compile-time effects of top-level forms

[sandra%orion@cs.utah.edu (Sandra J Loosemore)]

Issue:         COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS

References:    CLtL pages 66-70, 143

Category:      CLARIFICATION

Edit history:  V1, 07 Oct 1987 Sandra Loosemore (sandra@cs.utah.edu)
               V2, 15 Oct 1987 Sandra Loosemore (sandra@cs.utah.edu)

Related issues: none

Problem description:

CLtL currently leaves several issues unresolved regarding the compile-time
handling of top-level forms such as DEFMACRO and DEFVAR.  The purpose of
this proposal is to ensure that all CL implementations support usages
which appear to be standard programming practices.

This proposal addresses only the semantics of top-level defining forms
as it affects compilation of subsequent forms in the same file.  Specifically,
this proposal does *not* attempt to specify:

    (1) Compile-time behavior of defining forms which do not appear at 
        top-level within the file being compiled.

    (2) Inheritance or separation of the compiler environment from the normal,
        interpreter environment; or inheritance of the compiler environment
        across calls to COMPILE-FILE.


Proposal:

Certain defining macros, appearing at top-level within a file being
processed by COMPILE-FILE, normally have compile-time side effects which
affect how subsequent forms in the same file are compiled.  As an example,
DEFVAR should store information that tells the compiler that bindings to
the named variable that appear later in the file should be made special
bindings, rather than the default lexical bindings.

In some implementations, compile-time definitions may be handled
differently than those which would occur if the file was loaded in the
usual way.  It should not be assumed that the compile-time effects of the
defining forms remain in place after compilation is completed.  In
particular, an implementation may or may not make the definitions available
to the interpreter or to further calls to COMPILE-FILE.

The defining forms which have compile-time side effects are as follows:

DEFTYPE:   Type names defined via DEFTYPE must be recognized as valid in
subsequent type declarations.

DEFMACRO, DEFINE-MODIFY-MACRO:  Macro definitions must be stored at compile
time, so that occurences of the macro later on in the file will be expanded
correctly.  The body of the macro (*not* the expansion) must be evaluable
at compile time.

DEFUN:  An implementation may choose to store information about the
function for the purposes of compile-time error-checking (such as checking
the number of arguments on calls).  Portable code should not rely on DEFUN
making the function definition available at compile time.

DEFVAR, DEFPARAMETER:  The compiler must recognize that the variables
named by these forms have been proclaimed special.  The initial value form
must not be evaluated at compile time.

DEFCONSTANT:  An implementation may choose to store information about the
variable for the purposes of compile-time error-checking (such as checking
for rebinding of or assignment to the variable).  An implementation may also
choose to evaluate the inital value form at compile time.

DEFSETF, DEFINE-SETF-METHOD:  SETF methods must be available during the
expansion of calls to SETF later on in the file.  The body of
DEFINE-SETF-METHOD and the complex form of DEFSETF must be evaluable at
compile time.

DEFSTRUCT:  The structure type name must be recognized as a valid type name
in declarations, as for DEFTYPE.  The structure slot accessors must be made
known to SETF.  In addition, further DEFSTRUCT definitions should be able
to :INCLUDE a structure type defined earlier in the file being compiled.
The functions which DEFSTRUCT generates, and the #S reader syntax, may or
may not be available at compile time.


Test Case:

;;; DEFTYPE

(deftype small-int () '(int 0 10))

(let ((x 0))
     (declare (type (small-int x)))
     (format t "Type a number between 0 and 10:")
     (setq x (read))
     (format t "The number is ~s.~%" x))


;;; DEFMACRO

(defmacro silly (x) `(car ,x))

(let ((y  nil))
     (format t "Type a list:")
     (setq y (read))
     (format t "The SILLY macro returned ~s.~%" (silly y)))


;;; DEFVAR and DEFPARAMETER

(defvar *v1* 
    (progn 
	(print "Defvar should print this at load time.")
	'v1-global-value))
(defparameter *v2* 
    (progn
	(print "Defparameter should print this at load time.")
	'v2-global-value))

(defun test-defvar-and-defparameter ()
    (format t "*v1* = ~s~%" *v1*)
    (format t "*v2* = ~s~%" *v2*))

(let ((*v1*  'v1-special-binding-value)
      (*v2*  'v2-special-binding-value))
     (test-defvar-and-defparameter))


;;; DEFCONSTANT

(defconstant *v3*
    (progn
	(print "This may be printed at compile-time as well as load time.")
	'v3-value))


;;; DEFSETF

(defsetf silly set-silly)

(defun set-silly (x value)
    (if (y-or-n-p "Do you want to change the CAR of ~s to ~s?" x value)
	(setf (car x) value))
    value)

(setf (silly (list 'a 'b 'c)) 'j-random-luser)


;;; DEFSTRUCT

(defstruct person name age sex)
(defstruct (astronaut (:include person) (:conc-name astro-))
    helmet-size
    (favorite-beverage 'tang))


Rationale:

The proposal reflects standard programming practices.  The primary purpose
of the proposal is to make an explicit statement that CL supports the
behavior that most programmers expect and many implementations already
provide.

The only reference to compile-time processing of defining forms in CLtL
appears to be in reference to DEFMACRO, on page 143, where it is stated
that macro definitions must be ``seen'' by the compiler before the first
use of the macro.


Current practice:

Many (probably most) Common Lisp implementations, including VaxLisp and
Lucid Lisp, are already largely in conformance.  

Kyoto Common Lisp is a notable offender.  By default, KCL evaluates *all*
top level forms as they are compiled, which is clearly in violation of the
behavior specified on p 69-70 of CLtL.  There is a flag to disable the
compile-time evaluation, but then macros such as DEFMACRO, DEFVAR, etc. do
not make their definitions available at compile-time either.


Adoption Cost:

The expansions of defining macros typically store information on property
lists or in hash tables, where it is later accessed by the compiler.  The
simplest way to achieve the required behavior is to modify the expansions
to wrap an (EVAL-WHEN (EVAL COMPILE LOAD) ...) around the forms which store
this information.


Cost of non-adoption:

The current vagueness in CLtL on compiler semantics can lead to unexpected
portability problems.  At least one person commented on the original
version of the proposal with, in effect, "Doesn't CLtL already *say* that
somewhere?!?"  The problem now is that what many people (even experienced
programmers) think are portable programming practices are actually not.  


Benefits:

Adoption of the proposal will provide more definite guidelines on how to
write programs that will compile correctly under all CL implementations.


Conversion Cost:

Minimal.


Esthetics:

Without a definite statement on the compile-time behavior of top-level
defining forms, programmers need to wrap an explicit EVAL-WHEN around all
such forms to guarantee consistent behavior across implementations.  It
would be cleaner to specify a default behavior that does what people seem
to expect anyway.


Discussion:

Reaction to an earlier version of this proposal on the CL mailing list was
overwhelmingly positive.  The current version incorporates a couple
additional suggestions that were made by others, notably Robert Kerns.

-------