issue COMPILE-ENVIRONMENT-CONSISTENCY, V3

[sandra%defun@cs.utah.edu (Sandra J Loosemore)]

This issue was distributed in draft form before the October meeting.  I've
incorporated the one suggestion I've received on it since then.  This is one
of the issues I'd like to send out on a mail ballot in about a month, so
if you have any serious problems with the writeup, please say so ASAP.

JonL, since you volunteered to act as a go-between with the CLOS people,
could you make sure that section 2h says the right things?

-Sandra



Issue:		COMPILE-ENVIRONMENT-CONSISTENCY
References:	CLtL p. 68-69, 143, 321
		RAM's "Compiler Cleanup Proposal #3"
Category:	CLARIFICATION
Edit History:   V1, 2 Sep 1988, Sandra Loosemore (initial draft)
		V2, 9 Sep 1988, Sandra Loosemore (incorporate suggestions)
		V3, 26 Oct 1988, Sandra Loosemore (add suggestion from Benson)


Problem Description:

CLtL does not clearly specify what aspects of the compiletime
environment the compiler (or other preprocessor) may "wire in" to code
being compiled.


Proposal COMPILE-ENVIRONMENT-CONSISTENCY:CLARIFY:

Common Lisp deliberately leaves unspecified the time at which certain
transformations, such as macro expansion, are performed within a
program.  While some implementations perform such transformations
concurrently with evaluation, it is also legitimate to perform
transformations during a preprocessing phase.  For example, an
implementation might choose to apply transformations at "function
promotion time" (i.e., transformations are applied once during
evaluation of a surrounding FUNCTION special form), or to completely
transform each top-level form and all of its subforms before
performing any evaluation.  User programs cannot portably depend upon
either the time of such transformations, or the number of times the
transformations are applied.

In all cases, however, compiling a program (with COMPILE or
COMPILE-FILE) provides a mechanism for forcing these transformations
to be applied and a guarantee that, once compiled, no further
transformations will be applied to that program.

In the discussion that follows, the term "compiler" is to be understood
to include other preprocessors as well.  Likewise, the "compiletime
environment" refers to the environment in which program transformations
occur, while "runtime" refers to the time the program is executed.


(1) The following information *must* be present in the compiletime
environment for the preprocessor to apply the correct transformations.
This information need not also be present in the runtime environment.

    (a) Macro definitions must be available in the compiletime environment.
	The compiler may assume that forms that are lists beginning with
	a symbol that does not name a macro or special form is a function
	call.  (This implies that SETF methods must also be available at
	compiletime.)

    (b) Special variables must be declared as such before they are bound.
	The compiler must treat any undeclared variable binding as a
	lexical binding.


(2) The compiler *may* "wire in" the following kinds of information
into the code it produces, if the information is present in the
compiletime environment.  However, since implementations are not
required to do this, user code must ensure that the information is
also defined consistently in the runtime environment.  Except as
noted, the behavior of user programs is unspecified in situations
where the information is defined differently at runtime than at
compiletime, since the user cannot depend upon which will take
precedence in a particular implementation.  In all cases, the absence
of the information at compiletime is not an error, but its presence
may enable the compiler to do a better job.

    (a) The compiler may assume that functions that are defined and
	declared INLINE in the compiletime environment will retain the
	same definitions at runtime.

    (b) The compiler may assume that, within a named function, a
	recursive call to a function of the same name refers to the
	same function, unless that function has been declared NOTINLINE.

    (c) COMPILE-FILE may assume that, in the absence of NOTINLINE
	declarations, a call within the file being compiled to a named
	function which is defined in that file refers to that function.
	(This permits "block compilation" of files.)  The behavior of
	the program is unspecified if functions are redefined individually 
	at runtime.

    (d) The compiler may assume that the signature (or "contract") of
	all built-in Common Lisp functions will not change.  In addition,
	the compiler may treat all built-in Common Lisp functions as if
	they had been proclaimed INLINE.

    (e) The compile may assume that the signature (or "contract") of
	functions with FTYPE information available will not change.  (See
	issue FUNCTION-TYPE-ARGUMENT-TYPE-SEMANTICS.)

    (f) The compiler may "wire in" the values of symbolic constants
	that have been defined with DEFCONSTANT in the compiletime
	environment.

    (g) Types that are defined with DEFTYPE or DEFSTRUCT can be assumed to
	retain the same definition in the runtime environment as in the
	compiletime environment.  (Note that it is not an error for an
	unknown type to appear in a declaration at compiletime, although
	it is reasonable for the compiler to emit a warning in such a
	case.)

    (h) The compiler may assume that a class name defined by DEFCLASS
	that is present in the compiletime environment will also be a
	class name at runtime, and that class will be an instance of the
	same metaclass.  There may be additional conformance requirements
	imposed by the metaclass, but there are none for STANDARD-CLASS.

    (i) The compiler may assume that if type declarations are present
	in the compiletime environment, the corresponding variables and 
	functions present in the runtime environment will actually be of
	those types; otherwise, the behavior of the program is undefined.


(3) The compiler *must not* make any additional assumptions about
consistency between the compiletime and runtime environments.  In 
particular:

    (a) The compiler may not assume that functions that are defined
	in the compiletime environment will retain the either the
	same definition or the same signature at runtime, except
	in situations (2a) through (2e) above.  It is, however,
	reasonable for the compiler to emit warning messages about
	calls to functions that are defined at compiletime, but where
	the wrong number or type of arguments are supplied.

    (b) The compiler may not signal an error if it sees a call to a
	function that is not defined at compiletime, since that function
	may be provided at runtime.  Again, it is permissible to emit
	a warning in these situations.

	

Rationale:

This proposal generally reflects current practice.


Current Practice:

I know of no compiler that does not implement the provisions of item (1).

For item (2), most compilers (including Lucid) optimize self-recursive
calls by default.  Most compilers also opencode data structure
accessors (such as CAR) at some level of optimization, and some code
much more complicated built-in functions inline as well.  VaxLisp, for
example, normally compiles MEMBER inline.  The Lucid compiler makes
use of type declarations to perform generic-to-specific
transformations on many arithmetic and sequence functions, which is
also a form of inlining.  KCL performs block compilation by default,
and Lucid does so under certain conditions.


Cost to implementors:

Unknown, but probably minor.


Cost to users:

Since most implementations appear to be largely in conformance with the
proposal, users should notice little difference.


Benefits:

The presence of a definite specification of what may happen when will
help users structure their programs so they will compile correctly in
all Common Lisp implementations.


Discussion:

Most of the discussion on this issue has been centered on the
terminology describing error situations for item (2).  In most cases
where there is an inconsistency between compile-time and run-time, the
results will be unpredictable but harmless.  The major exception is
violation of type declarations, which is a "crash-and-burn" error in
many implementations.

There has also been some concern raised that item (3) would prohibit
such things as a cross-compiler that produces standalone programs in
an environment that disallows redefinition of functions.  The intent
of this proposal is not to prohibit a compiler from having a magic
switch that imposes additional restrictions on the programs it
compiles, but such a compiler would not be a compiler for Common Lisp.
-------