issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS

[sandra@cs.utah.edu (Sandra J Loosemore)]

Issue:		COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
References:	CLtL pages 66-70, 143
Category:	CLARIFICATION
Edit history:   V1, 07 Oct 1987 Sandra Loosemore
                V2, 15 Oct 1987 Sandra Loosemore
                V3, 15 Jan 1988 Sandra Loosemore
		V4, 06 May 1988 Sandra Loosemore



Problem Description:

Standard programming practices assume that, when calls to defining
macros such as DEFMACRO and DEFVAR are compiled, certain side-effects
occur that affect how the compiler processes subsequent forms.
However, these side-effects are not mentioned in CLtL, except for a
passing mention that macro definitions must be ``seen'' by the
compiler before it can compile calls to those macros correctly. 


Proposal: COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS:CLARIFY

(1) Certain defining macros, appearing within a file being processed
by COMPILE-FILE, normally have compile-time side effects which affect
how subsequent forms in the same file are compiled.  The defining macros
and their specific side effects are as follows:
 
DEFTYPE:   Type names defined via DEFTYPE must be recognized as valid in
subsequent type declarations.
 
DEFMACRO, DEFINE-MODIFY-MACRO:  Macro definitions must be stored at compile
time, so that occurences of the macro later on in the file will be expanded
correctly.  The body of the macro (but not necesarily its expansion) must 
be evaluable at compile time.
 
DEFUN:  An implementation may choose to store information about the
function for the purposes of compile-time error-checking (such as checking
the number of arguments on calls).  Portable code should not rely on DEFUN
making the function definition available at compile time.
 
DEFVAR, DEFPARAMETER:  The compiler must recognize that the variables
named by these forms have been proclaimed special.  The initial value form
must not be evaluated at compile time.
 
DEFCONSTANT:  An implementation may choose to store information about the
variable for the purposes of compile-time error-checking (such as checking
for rebinding of or assignment to the variable).  If the initial value form
is a constant, an implementation may also choose to evaluate it at compile 
time for the purposes of constant-folding.
 
DEFSETF, DEFINE-SETF-METHOD:  SETF methods must be available during the
expansion of calls to SETF later on in the file.  The body of
DEFINE-SETF-METHOD and the complex form of DEFSETF must be evaluable at
compile time.
 
DEFSTRUCT:  The structure type name must be recognized as a valid type name
in declarations, as for DEFTYPE.  The structure slot accessors must be made
known to SETF.  In addition, further DEFSTRUCT definitions should be able
to :INCLUDE a structure type defined earlier in the file being compiled.
The functions which DEFSTRUCT generates, and the #S reader syntax, may or
may not be available at compile time.

(2) A model which explains how these compile-time side-effects happen is
that each defining macro expands into one or more EVAL-WHEN forms.  The
bodies of the EVAL-WHEN forms contain code which causes the appropriate
information to be stored.  Different information may be stored at
compile-time than when the defining macros are processed interpretively
or when the compiled file is loaded.

(3) The information stored by the defining macros at compile time may
or may not be available to the interpreter (either during or after
compilation), or during subsequent calls to COMPILE or COMPILE-FILE.
For example, the following code is nonportable because it assumes that the
compiler stores the macro definition of FOO where it is available to the
interpreter:

    (defmacro foo (x) `(car ,x))
    (eval-when (eval compile load)
        (print (foo '(a b c))))

A portable way to do the same thing would be to include the macro definition
inside the EVAL-WHEN:

    (eval-when (eval compile load)
        (defmacro foo (x) `(car ,x))
        (print (foo '(a b c))))



Rationale:

The proposal reflects standard programming practices.  The primary purpose
of the proposal is to make an explicit statement that CL supports the
behavior that most programmers expect and many implementations already
provide.


Current Practice:

Many (probably most) Common Lisp implementations, including VaxLisp and
Lucid Lisp, are already largely in conformance.  
 
Kyoto Common Lisp is a notable offender.  By default, KCL evaluates *all*
top level forms as they are compiled, which is clearly in violation of the
behavior specified on p 69-70 of CLtL.  There is a flag to disable the
compile-time evaluation, but then macros such as DEFMACRO, DEFVAR, etc. do
not make their definitions available at compile-time either.


Cost to implementors:

Making the defining macros expand into EVAL-WHENs to store the required
information is a simple and recommended implementation technique.


Cost to users:

Since CLtL does not specify whether and what compile-time side-effects
happen, any user code which relies on them is, strictly speaking,
nonportable.  In practice, however, most programmers already expect
the behavior described in this proposal and will not find it to be
an incompatible change.


Benefits:

Adoption of the proposal will provide more definite guidelines on how to
write programs that will compile correctly under all CL implementations.


Discussion:

Reaction to an earlier version of this proposal on the CL mailing list was
overwhelmingly positive.

The only major criticism of this proposal I have heard is that it should
also include PROCLAIM.  However, since PROCLAIM is not a macro, its
compile-time side effects cannot be handled using the same mechanism.
A separate proposal seems more appropriate.

-------