[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
issue EVAL-WHEN-NON-TOP-LEVEL, version 6
- To: cl-compiler@sail.stanford.edu
- Subject: issue EVAL-WHEN-NON-TOP-LEVEL, version 6
- From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
- Date: Thu, 9 Mar 89 17:04:08 MST
[How did the mailer lose the text to this message when I sent it out
before?]
With the help of Pitman and Moon, I have cleaned up the presentation
of the GENERALIZE-EVAL proposal from version 5. Among other things, I
have moved the definition of "top-level" here (from issue
DEFINING-MACROS-NON-TOP-LEVEL), so there is now a more coherent
description of how the compiler processes top-level forms. (If
anybody has been looking over the working draft of the standard, that
description is going to end up in section 4.2.)
Issue: EVAL-WHEN-NON-TOP-LEVEL
Forum: Compiler
References: EVAL-WHEN (CLtL pp69-70),
Issue DEFINING-MACROS-NON-TOP-LEVEL
Issue COMPILED-FUNCTION-REQUIREMENTS
Issue IN-PACKAGE-FUNCTIONALITY
Issue LOCALLY-TOP-LEVEL
Category: CLARIFICATION/CHANGE
Edit History: 06-May-88, Version 1 by Sandra Loosemore
16-Dec-88, Version 2 by Loosemore (alternate direction)
30-Dec-88, Version 3 by Loosemore (minor wording changes)
07-Jan-89, Version 4 by Loosemore (update discussion)
09-Feb-89, Version 5 by Pitman and Moon (some major changes)
09-Mar-89, Version 6 by Loosemore (clean up wording)
Status: Ready for release
Problem Description:
The current description of how the compiler should handle EVAL-WHEN
only makes sense when it appears as a top-level form in the file being
compiled. Is it legitimate for EVAL-WHEN to appear in non-top-level
locations? Even if it is legitimate, what does it mean?
Another issue, referred to here as ``the EVAL-WHEN shadowing problem,''
is that some people have complained that shadowing the symbols EVAL,
COMPILE, or LOAD means that you have to also either shadow EVAL-WHEN
and define it to recognize the new symbol, or else you must resign
yourself to writing (EVAL-WHEN (... LISP:EVAL ...) ...),etc. all over.
While the goal here is not to solve this problem, it might be possible
to solve both problems at once.
There are two proposals presented here, GENERALIZE-EVAL and
GENERALIZE-EVAL-NEW-KEYWORDS.
Background/Analysis:
The proposal which follows was constructed with the following goals
in mind:
1. The lexical and dynamic environment for the EVAL-WHEN body should
be the same for each situation. That is, the body should ``mean
the same thing'' regardless of which situation is being processed.
2. The evaluation context for EVAL-WHEN should be the current
lexical environment.
3. At execution time, EVAL-WHEN should always return the result of
its last form if execution of the body occurred, or NIL if the
body was not executed.
4. If a top-level EVAL-WHEN has a LOAD keyword, its body should
inherit top-level-ness during normal processing. This permits the
use of (EVAL-WHEN (EVAL COMPILE LOAD) ...) at top-level to mean
simply "Do whatever would normally be done for this body, but
also do something at compile time." This, in turn, will later be
the key to allowing defining forms to be usefully described in
terms of EVAL-WHEN.
5. Non-top-level expressions should have no effect until they are
executed. This is the key to making sure that any necessary
environment is present. Since the COMPILE keyword forces effects
to occur earlier than execution time, it follows from this that
any correct solution must not allow the COMPILE keyword to have
an effect at other than top-level.
To accomplish these goals, we formulated the following model:
The purpose of EVAL-WHEN is to accomodate the fact that some of the
semantic processing of an expression may usefully be partitioned
between compile time and run time in some circumstances.
(EVAL-WHEN (EVAL) <code>)
describes a general technique for accomplishing some particular goal
at normal program execution time. However, the pair of expressions
(EVAL-WHEN (COMPILE) <code-A>)
(EVAL-WHEN (LOAD) <code-B>)
can be used to describe an alternate technique for implementing part
of the effect (A) at compile-time, and part of the effect (B) at
load-time.
Proposal (EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL):
Replace the description of EVAL-WHEN with the following:
EVAL-WHEN ({situation}*) {form}* [Special Form]
The body of an EVAL-WHEN form is processed as an implicit PROGN, but
only in the situations listed. Each SITUATION must be a symbol,
either COMPILE, LOAD, or EVAL.
The use of COMPILE and LOAD controls whether and when processing
occurs for top-level forms. The use of EVAL controls whether
processing occurs for non-top-level forms.
The EVAL-WHEN construct may be more precisely understood in terms of
a model of how the file compiler, COMPILE-FILE, processes forms in a
file to be compiled.
Successive forms are read from the file by the file compiler using
READ. These top-level forms are normally processed in what we call
`not-compile-time' mode. There is one other mode, called
`compile-time-too' mode, which can come into play for top-level
forms. The EVAL-WHEN special form is used to annotate a program
in a way that allows the program doing the processing to select
the appropriate mode.
Processing of top-level forms in the file compiler works as follows:
* If the form is a macro call, it is expanded and the result is
processed as a top-level form in the same processing mode
(compile-time-too or not-compile-time).
* If the form is a PROGN form, each of its body forms is
sequentially processed as top-level forms in the same processing
mode.
* If the form is a COMPILER-LET, MACROLET, or SYMBOL-MACROLET,
the file compiler makes the appropriate bindings and recursively
processes the body forms as an implicit top-level PROGN with those
bindings in effect, in the same processing mode.
* If the form is an EVAL-WHEN form, it is handled according to
the following table:
COMPILE LOAD EVAL compile-time-too Action
Yes Yes -- -- Process body in compile-time-too mode
No Yes Yes Yes Process body in compile-time-too mode
No Yes Yes No Process body in not-compile-time mode
No Yes No -- Process body in not-compile-time mode
Yes No -- -- Evaluate body
No No Yes Yes Evaluate body
No No Yes No do nothing
No No No -- do nothing
"Process body" means to process the body as an implicit top-level
PROGN. "Evaluate body" means to evaluate the body forms as in
implicit PROGN in the dynamic execution context of the compiler and
in the lexical environment in which the EVAL-WHEN appears.
* Otherwise, the form is a top-level form that is not one of the
special cases. If in compile-time-too mode, the compiler first
evaluates the form and then performs normal compiler processing
on it. If in not-compile-time mode, only normal compiler
processing is performed. [The nature of this processing is
defined more precisely in issue COMPILED-FUNCTION-REQUIREMENTS.]
Any subforms are treated as non-top-level forms.
For an EVAL-WHEN form that is not a top-level form in the file compiler
(that is, one of: in the interpreter; in COMPILE; or in the file
compiler but not at top-level), if the EVAL situation is specified,
its body is treated as an implicit PROGN. Otherwise, the EVAL-WHEN
form returns NIL.
Clarifications/Consequences:
The following effects are logical consequences of the above proposal:
* It is never the case that the execution of a single EVAL-WHEN
expression will execute the body code more than once.
* The keyword `EVAL' is a misnomer because execution of
the body need not be done by EVAL. In compiled code, such as
(DEFUN FOO () (EVAL-WHEN (EVAL) (PRINT 'FOO)))
the call to PRINT should be compiled.
* Macros intended for use in top-level forms should arrange for all
side-effects to be done by the forms in the macro expansion.
The macro-expander itself should not do the side-effects.
Wrong: (defmacro foo ()
(really-foo)
`(really-foo))
Right: (defmacro foo ()
`(eval-when (compile eval load) (really-foo)))
Adherence to this convention will mean that such macros will behave
intuitively when placed in non-top-level positions.
* Placing a variable binding around an EVAL-WHEN reliably captures the
binding because the `compile-time-too' mode cannot occur (because
introducing a variable binding would mean we were not at top level).
For example,
(LET ((X 3))
(EVAL-WHEN (EVAL LOAD COMPILE) (PRINT X)))
will print 3 at execution [load] time, and will not print anything at
compile time. This is important so that expansions of DEFUN and
DEFMACRO can be done in terms of EVAL-WHEN and can correctly capture
the lexical environment.
(DEFUN BAR (X) (DEFUN FOO () (+ X 3)))
might expand into
(DEFUN BAR (X)
(PROGN (EVAL-WHEN (COMPILE)
(COMPILER::NOTICE-FUNCTION-DEFINITION 'FOO '(X)))
(EVAL-WHEN (EVAL LOAD)
(SETF (SYMBOL-FUNCTION 'FOO) #'(LAMBDA () (+ X 3))))))
which would be treated the same as
(DEFUN BAR (X)
(SETF (SYMBOL-FUNCTION 'FOO) #'(LAMBDA () (+ X 3))))
by the above rules.
Test Cases:
;; #1: The EVAL-WHEN in this case is not at top-level, so only the EVAL
;; keyword is considered. At compile time, this has no effect.
;; At load time (if the LET is at top level), or at execution time
;; (if the LET is embedded in some other form which does not execute
;; until later) this sets (SYMBOL-FUNCTION 'FOO1) to a function which
;; returns 1.
(LET ((X 1))
(EVAL-WHEN (EVAL LOAD COMPILE)
(SETF (SYMBOL-FUNCTION 'FOO1) #'(LAMBDA () X))))
;; #2: If this expression occurs at the top-level of a file to be compiled,
;; it has BOTH a compile time AND a load-time effect of setting
;; (SYMBOL-FUNCTION 'FOO2) to a function which returns 2.
(EVAL-WHEN (EVAL LOAD COMPILE)
(LET ((X 2))
(EVAL-WHEN (EVAL LOAD COMPILE)
(SETF (SYMBOL-FUNCTION 'FOO2) #'(LAMBDA () X)))))
;; #3: If this expression occurs at the top-level of a file to be compiled,
;; it has BOTH a compile time AND a load-time effect of setting the
;; function cell of FOO3 to a function which returns 3.
(EVAL-WHEN (EVAL LOAD COMPILE)
(SETF (SYMBOL-FUNCTION 'FOO3) #'(LAMBDA () 3)))
;; #4: This always does nothing. It simply returns NIL.
(EVAL-WHEN (COMPILE)
(EVAL-WHEN (COMPILE)
(PRINT 'FOO4)))
;; #5: If this form occurs at top-level of a file to be compiled, FOO5 is
;; printed at compile time. If this form occurs in a non-top-level
;; position, nothing is printed at compile time. Regardless of context,
;; nothing is ever printed at load time or execution time.
(EVAL-WHEN (COMPILE)
(EVAL-WHEN (EVAL)
(PRINT 'FOO5)))
;; #6: If this form occurs at top-level of a file to be compiled, FOO6 is
;; printed at compile time. If this form occurs in a non-top-level
;; position, nothing is printed at compile time. Regardless of context,
;; nothing is ever printed at load time or execution time.
(EVAL-WHEN (EVAL LOAD)
(EVAL-WHEN (COMPILE)
(PRINT 'FOO6)))
Rationale:
This is compatible with any guarantees made by CLtL, and extends the
behavior usefully to non-top-level situations.
This gives a useful meaning to EVAL-WHEN that supports useful and
predictable behavior if defining macros are used in a non-top-level
situation.
Proposal (EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL-NEW-KEYWORDS):
As in GENERALIZE-EVAL, but rename the EVAL keyword to :EXECUTE,
the COMPILE keyword to :COMPILE-TOPLEVEL, and LOAD keyword to
:LOAD-TOPLEVEL.
Deprecate the use of keywords EVAL, COMPILE, and LOAD to EVAL-WHEN.
For compatibility, they are supported in EVAL-WHEN
at top-level, but their meaning is not defined elsewhere.
Rationale:
The fact that the situation keywords chosen are not the same as
those now used means that the change can be added in a way that
is truly upward compatible (not only with CLtL but with existing
practice in implementations which have chosen to extend or `clarify'
the definition given in CLtL) since the meaning of EVAL, COMPILE,
and LOAD in non-top-level situations (which was never spelled
out in CLtL) can legitimately differ from the meaning of these
new keywords.
Using other names and/or the keyword package for the names of
situations solves the EVAL-WHEN shadowing problem.
The name `execute' does not promote the confusion that the body of an
EVAL-WHEN must be executed only in the evaluator. It also does not
promote the confusion that the body of an EVAL-WHEN, regardless of when
executed, must run interpreted.
The names `compile-toplevel' and `load-toplevel' emphasize the fact
that these cases are not interesting in non-top-level positions.
Current Practice:
In Symbolics Genera, the interpreter permits EVAL-WHEN in non-top-level
positions in a way that is compatible with this proposal but both the
COMPILE and COMPILE-FILE functions complain about EVAL-WHEN in a
non-top-level position.
Both Lucid Common Lisp and Kyoto Common Lisp already interpret the
EVAL keyword to mean "execute" in non-top-level situations. Both of
these implementations also make (EVAL-WHEN (LOAD) ...) suppress
compile-time "magic" from defining macros such as DEFMACRO.
IIM describes its EVAL-WHEN as:
(defmacro eval-when (situations &body body &environment env)
(if (not (compiler-environment-p env))
(when (member 'eval situations) `(progn ,@body))
(progn
(when (member 'compile situations)
(if (compiler-at-top-level-p env)
(mapc #'eval body)
(warn "Top-level form encountered at non-top-level.")))
(when (member 'load situations) `(progn ,@body)))))
Note that the interpretation of the EVAL situation and the nesting
behavior is different.
Cost to Implementors:
The actual change to EVAL-WHEN in both cases is probably fairly
localized and straightforward to make in most or all implementations.
The second-order costs of proposal GENERALIZE-EVAL will vary depending
on whether existing implementations have extended the definition of
EVAL-WHEN in incompatible ways. If an implementation has made such
extensions, there may be user and system code which depends on them
and the cost of converting that code may be non-trivial. There is
presumably also documentation impact.
Proposal GENERALIZE-EVAL-NEW-KEYWORDS avoids most or all of the
second-order costs of proposal GENERALIZE-EVAL.
The compiler processing for top-level forms might be implemented
something like:
;;; Forms read by the file compiler are passed to PROCESS-TOP-LEVEL-FORM
;;; with a env compile-time-too both NIL.
(defun process-top-level-form (form env compile-time-too)
(setq form (macroexpand form env))
(cond ((not (consp form))
nil)
((eq (car form) 'progn)
(dolist (f (cdr form))
(process-top-level-form f env compile-time-too)))
((eq (car form) 'compiler-let)
(process-compiler-let form env compile-time-too))
((eq (car form) 'macrolet)
(process-macrolet form env compile-time-too))
((eq (car form) 'symbol-macrolet)
(process-symbol-macrolet form env compile-time-too))
((eq (car form) 'eval-when)
(process-eval-when form env compile-time-too))
(t
(if compile-time-too
(internal-eval form env))
(compile-form form env))
))
(defun process-eval-when (form env compile-time-too)
(let* ((situations (cadr form))
(body (cddr form))
(compile-p (member 'compile situations))
(load-p (member 'load situations))
(eval-p (member 'eval situations)))
(cond ((or (and compile-p load-p)
(and eval-p load-p compile-time-too))
(process-top-level-form `(progn ,@body) env t))
(load-p
(process-top-level-form `(progn ,@body) env nil))
((or compile-p
(and eval-p compile-time-too))
(dolist (f body)
(internal-eval f env)))
(t
nil))))
;;; PROCESS-COMPILER-LET, PROCESS-MACROLET, and PROCESS-SYMBOL-MACROLET
;;; do the obvious things.
;;; INTERNAL-EVAL evaluates "form" in lexical environment "env".
Cost to Users:
Technically, none. Either proposal is technically upward compatible
with CLtL.
Proposal GENERALIZE-EVAL might force some extended implementations to
change incompatibly. As such, some users who depend on
implementation-dependent extensions might have to adjust their code
somewhat to deal with those changes.
Proposal GENERALIZE-EVAL-NEW-KEYWORDS does not force implementations
to change incompatibly, so has no forced impact on users.
Cost of Non-Adoption:
EVAL-WHEN is a mess. Using it as the low-level substrate into which
defining macros should expand, and guaranteeing any predictable effects
of those macros in non-top-level situations is currently difficult and
would continue to be so in the absence of some resolution on this issue.
Benefits:
The costs of non-adoption would be avoided: it would be possible to
use EVAL-WHEN in many situations where it cannot currently be used
reliably.
The portability of many existing tools which use EVAL-WHEN internally
in macros will be enhanced.
Aesthetics:
This generalization of the meaning makes the purpose and uses of
EVAL-WHEN less mysterious. In that sense, aesthetics are simplified
somewhat.
Discussion:
The cleanup issue LOCALLY-TOP-LEVEL would make LOCALLY also "pass
through" top-level-ness to its body. The reason why that is not
addressed in this issue is that it involves making LOCALLY a special
form.
Pitman and Moon don't care whether we say `top level,' `top-level,' or
`toplevel.' The spelling choices in this writeup are arbitrary. If
necessary, the proposal GENERALIZE-EVAL-NEW-KEYWORDS could be amended
to propose :COMPILE-TOP-LEVEL, etc.
Pitman, Moon, and Bob Laddaga (a Symbolics Cloe implementor) support
both of these proposals. Pitman and Laddaga have a preference for
GENERALIZE-EVAL-NEW-KEYWORDS. Moon is neutral about which should be
preferred.
Sandra Loosemore says:
I still feel somewhat uncomfortable with the definition of EVAL-WHEN
presented here, mostly because its nesting behavior is so unintuitive
(as in test case number 6). We have also had a hard time in deciding
what the term "top-level" really means; the definition presented here
is rather arbitrary. However, since we have run out of time in which
to come up with acceptable alternatives, I'm willing to go along with
proposal GENERALIZE-EVAL. It is compatible with the description in
CLtL but presented in a more coherent way, and I think it is an
improvement. On the other hand, I don't really like the idea of
changing the names of the keywords; if we are going to make an
incompatible change, the right thing to do would be to throw out
EVAL-WHEN entirely and start from scratch.
-------