[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

issue EVAL-WHEN-NON-TOP-LEVEL, version 6

[How did the mailer lose the text to this message when I sent it out

With the help of Pitman and Moon, I have cleaned up the presentation
of the GENERALIZE-EVAL proposal from version 5.  Among other things, I
have moved the definition of "top-level" here (from issue
DEFINING-MACROS-NON-TOP-LEVEL), so there is now a more coherent
description of how the compiler processes top-level forms.  (If
anybody has been looking over the working draft of the standard, that
description is going to end up in section 4.2.)

Forum:        Compiler
References:   EVAL-WHEN (CLtL pp69-70),
Edit History: 06-May-88, Version 1 by Sandra Loosemore
              16-Dec-88, Version 2 by Loosemore (alternate direction)
              30-Dec-88, Version 3 by Loosemore (minor wording changes)
              07-Jan-89, Version 4 by Loosemore (update discussion)
              09-Feb-89, Version 5 by Pitman and Moon (some major changes)
	      09-Mar-89, Version 6 by Loosemore (clean up wording)
Status:       Ready for release

Problem Description:

  The current description of how the compiler should handle EVAL-WHEN
  only makes sense when it appears as a top-level form in the file being
  compiled. Is it legitimate for EVAL-WHEN to appear in non-top-level
  locations? Even if it is legitimate, what does it mean?
  Another issue, referred to here as ``the EVAL-WHEN shadowing problem,''
  is that some people have complained that shadowing the symbols EVAL,
  COMPILE, or LOAD means that you have to also either shadow EVAL-WHEN
  and define it to recognize the new symbol, or else you must resign
  yourself to writing (EVAL-WHEN (... LISP:EVAL ...) ...),etc. all over.
  While the goal here is not to solve this problem, it might be possible
  to solve both problems at once.

  There are two proposals presented here, GENERALIZE-EVAL and


  The proposal which follows was constructed with the following goals
  in mind:

    1. The lexical and dynamic environment for the EVAL-WHEN body should
       be the same for each situation.  That is, the body should ``mean
       the same thing'' regardless of which situation is being processed.

    2. The evaluation context for EVAL-WHEN should be the current
       lexical environment.

    3. At execution time, EVAL-WHEN should always return the result of
       its last form if execution of the body occurred, or NIL if the
       body was not executed.

    4. If a top-level EVAL-WHEN has a LOAD keyword, its body should 
       inherit top-level-ness during normal processing. This permits the
       use of (EVAL-WHEN (EVAL COMPILE LOAD) ...) at top-level to mean
       simply "Do whatever would normally be done for this body, but
       also do something at compile time." This, in turn, will later be
       the key to allowing defining forms to be usefully described in
       terms of EVAL-WHEN.

    5. Non-top-level expressions should have no effect until they are
       executed. This is the key to making sure that any necessary
       environment is present. Since the COMPILE keyword forces effects
       to occur earlier than execution time, it follows from this that
       any correct solution must not allow the COMPILE keyword to have
       an effect at other than top-level.

  To accomplish these goals, we formulated the following model:

    The purpose of EVAL-WHEN is to accomodate the fact that some of the
    semantic processing of an expression may usefully be partitioned
    between compile time and run time in some circumstances.

    (EVAL-WHEN (EVAL) <code>)
    describes a general technique for accomplishing some particular goal
    at normal program execution time. However, the pair of expressions
    (EVAL-WHEN (COMPILE) <code-A>)
    (EVAL-WHEN (LOAD) <code-B>)
    can be used to describe an alternate technique for implementing part
    of the effect (A) at compile-time, and part of the effect (B) at


  Replace the description of EVAL-WHEN with the following:

  EVAL-WHEN ({situation}*) {form}*                      [Special Form]

  The body of an EVAL-WHEN form is processed as an implicit PROGN, but
  only in the situations listed.  Each SITUATION must be a symbol,
  either COMPILE, LOAD, or EVAL.

  The use of COMPILE and LOAD controls whether and when processing
  occurs for top-level forms. The use of EVAL controls whether
  processing occurs for non-top-level forms.

  The EVAL-WHEN construct may be more precisely understood in terms of
  a model of how the file compiler, COMPILE-FILE, processes forms in a
  file to be compiled.

  Successive forms are read from the file by the file compiler using 
  READ. These top-level forms are normally processed in what we call
  `not-compile-time' mode. There is one other mode, called 
  `compile-time-too' mode, which can come into play for top-level
  forms. The EVAL-WHEN special form is used to annotate a program
  in a way that allows the program doing the processing to select
  the appropriate mode.

  Processing of top-level forms in the file compiler works as follows:

   * If the form is a macro call, it is expanded and the result is
     processed as a top-level form in the same processing mode
     (compile-time-too or not-compile-time).

   * If the form is a PROGN form, each of its body forms is
     sequentially processed as top-level forms in the same processing

     the file compiler makes the appropriate bindings and recursively
     processes the body forms as an implicit top-level PROGN with those 
     bindings in effect, in the same processing mode.

   * If the form is an EVAL-WHEN form, it is handled according to
     the following table:

     COMPILE LOAD EVAL compile-time-too Action
       Yes   Yes  --     --             Process body in compile-time-too mode
       No    Yes  Yes    Yes            Process body in compile-time-too mode
       No    Yes  Yes    No             Process body in not-compile-time mode
       No    Yes  No     --             Process body in not-compile-time mode
       Yes   No   --     --             Evaluate body
       No    No   Yes    Yes            Evaluate body
       No    No   Yes    No             do nothing
       No    No   No     --             do nothing

     "Process body" means to process the body as an implicit top-level
     PROGN.  "Evaluate body" means to evaluate the body forms as in
     implicit PROGN in the dynamic execution context of the compiler and
     in the lexical environment in which the EVAL-WHEN appears.

   * Otherwise, the form is a top-level form that is not one of the
     special cases.  If in compile-time-too mode, the compiler first
     evaluates the form and then performs normal compiler processing
     on it.  If in not-compile-time mode, only normal compiler
     processing is performed.  [The nature of this processing is
     defined more precisely in issue COMPILED-FUNCTION-REQUIREMENTS.]
     Any subforms are treated as non-top-level forms.

  For an EVAL-WHEN form that is not a top-level form in the file compiler
  (that is, one of: in the interpreter; in COMPILE; or in the file
  compiler but not at top-level), if the EVAL situation is specified,
  its body is treated as an implicit PROGN.  Otherwise, the EVAL-WHEN
  form returns NIL.


  The following effects are logical consequences of the above proposal:

   * It is never the case that the execution of a single EVAL-WHEN
     expression will execute the body code more than once.

   * The keyword `EVAL' is a misnomer because execution of
     the body need not be done by EVAL. In compiled code, such as
     the call to PRINT should be compiled.

   * Macros intended for use in top-level forms should arrange for all
     side-effects to be done by the forms in the macro expansion.
     The macro-expander itself should not do the side-effects.

       Wrong:  (defmacro foo ()
       Right:  (defmacro foo ()
                 `(eval-when (compile eval load) (really-foo)))

     Adherence to this convention will mean that such macros will behave
     intuitively when placed in non-top-level positions.

   * Placing a variable binding around an EVAL-WHEN reliably captures the
     binding because the `compile-time-too' mode cannot occur (because 
     introducing a variable binding would mean we were not at top level).
     For example,

        (LET ((X 3))

     will print 3 at execution [load] time, and will not print anything at
     compile time.  This is important so that expansions of DEFUN and 
     DEFMACRO can be done in terms of EVAL-WHEN and can correctly capture
     the lexical environment.

        (DEFUN BAR (X) (DEFUN FOO () (+ X 3)))

     might expand into

        (DEFUN BAR (X) 
                 (EVAL-WHEN (EVAL LOAD)
                   (SETF (SYMBOL-FUNCTION 'FOO) #'(LAMBDA () (+ X 3))))))

     which would be treated the same as

        (DEFUN BAR (X) 
          (SETF (SYMBOL-FUNCTION 'FOO) #'(LAMBDA () (+ X 3))))

     by the above rules.

 Test Cases:

  ;; #1: The EVAL-WHEN in this case is not at top-level, so only the EVAL
  ;;     keyword is considered. At compile time, this has no effect.
  ;;     At load time (if the LET is at top level), or at execution time
  ;;     (if the LET is embedded in some other form which does not execute
  ;;     until later) this sets (SYMBOL-FUNCTION 'FOO1) to a function which
  ;;     returns 1.
  (LET ((X 1))
  ;; #2: If this expression occurs at the top-level of a file to be compiled,
  ;;     it has BOTH a compile time AND a load-time effect of setting
  ;;     (SYMBOL-FUNCTION 'FOO2) to a function which returns 2.
    (LET ((X 2))
        (SETF (SYMBOL-FUNCTION 'FOO2) #'(LAMBDA () X)))))
  ;; #3: If this expression occurs at the top-level of a file to be compiled,
  ;;     it has BOTH a compile time AND a load-time effect of setting the
  ;;     function cell of FOO3 to a function which returns 3.
  ;; #4: This always does nothing. It simply returns NIL.
      (PRINT 'FOO4)))
  ;; #5: If this form occurs at top-level of a file to be compiled, FOO5 is
  ;;     printed at compile time. If this form occurs in a non-top-level
  ;;     position, nothing is printed at compile time. Regardless of context,
  ;;     nothing is ever printed at load time or execution time.
      (PRINT 'FOO5)))

  ;; #6: If this form occurs at top-level of a file to be compiled, FOO6 is
  ;;     printed at compile time.  If this form occurs in a non-top-level
  ;;     position, nothing is printed at compile time. Regardless of context,
  ;;     nothing is ever printed at load time or execution time.

      (PRINT 'FOO6)))
  This is compatible with any guarantees made by CLtL, and extends the
  behavior usefully to non-top-level situations.

  This gives a useful meaning to EVAL-WHEN that supports useful and
  predictable behavior if defining macros are used in a non-top-level


  As in GENERALIZE-EVAL, but rename the EVAL keyword to :EXECUTE,
  the COMPILE keyword to :COMPILE-TOPLEVEL, and LOAD keyword to 

  Deprecate the use of keywords EVAL, COMPILE, and LOAD to EVAL-WHEN.
  For compatibility, they are supported in EVAL-WHEN
  at top-level, but their meaning is not defined elsewhere.


  The fact that the situation keywords chosen are not the same as
  those now used means that the change can be added in a way that
  is truly upward compatible (not only with CLtL but with existing
  practice in implementations which have chosen to extend or `clarify'
  the definition given in CLtL) since the meaning of EVAL, COMPILE,
  and LOAD in non-top-level situations (which was never spelled
  out in CLtL) can legitimately differ from the meaning of these
  new keywords.

  Using other names and/or the keyword package for the names of
  situations solves the EVAL-WHEN shadowing problem.

  The name `execute' does not promote the confusion that the body of an
  EVAL-WHEN must be executed only in the evaluator. It also does not
  promote the confusion that the body of an EVAL-WHEN, regardless of when
  executed, must run interpreted.

  The names `compile-toplevel' and `load-toplevel' emphasize the fact
  that these cases are not interesting in non-top-level positions.

Current Practice:

  In Symbolics Genera, the interpreter permits EVAL-WHEN in non-top-level 
  positions in a way that is compatible with this proposal but both the
  COMPILE and COMPILE-FILE functions complain about EVAL-WHEN in a
  non-top-level position.

  Both Lucid Common Lisp and Kyoto Common Lisp already interpret the
  EVAL keyword to mean "execute" in non-top-level situations.  Both of
  these implementations also make (EVAL-WHEN (LOAD) ...) suppress
  compile-time "magic" from defining macros such as DEFMACRO.

  IIM describes its EVAL-WHEN as:
   (defmacro eval-when (situations &body body &environment env)
     (if (not (compiler-environment-p env))
         (when (member 'eval situations) `(progn ,@body))
           (when (member 'compile situations)
             (if (compiler-at-top-level-p env)
                 (mapc #'eval body)
                 (warn "Top-level form encountered at non-top-level.")))
           (when (member 'load situations) `(progn ,@body)))))
  Note that the interpretation of the EVAL situation and the nesting
  behavior is different.

Cost to Implementors:

  The actual change to EVAL-WHEN in both cases is probably fairly
  localized and straightforward to make in most or all implementations.

  The second-order costs of proposal GENERALIZE-EVAL will vary depending
  on whether existing implementations have extended the definition of
  EVAL-WHEN in incompatible ways. If an implementation has made such
  extensions, there may be user and system code which depends on them
  and the cost of converting that code may be non-trivial. There is
  presumably also documentation impact.

  Proposal GENERALIZE-EVAL-NEW-KEYWORDS avoids most or all of the 
  second-order costs of proposal GENERALIZE-EVAL.

  The compiler processing for top-level forms might be implemented 
  something like:

  ;;; Forms read by the file compiler are passed to PROCESS-TOP-LEVEL-FORM
  ;;;    with a env compile-time-too both NIL.
  (defun process-top-level-form (form env compile-time-too)
      (setq form (macroexpand form env))
      (cond ((not (consp form))
            ((eq (car form) 'progn)
             (dolist (f (cdr form))
                 (process-top-level-form f env compile-time-too)))
            ((eq (car form) 'compiler-let)
             (process-compiler-let form env compile-time-too))
            ((eq (car form) 'macrolet)
             (process-macrolet form env compile-time-too))
            ((eq (car form) 'symbol-macrolet)
             (process-symbol-macrolet form env compile-time-too))
            ((eq (car form) 'eval-when)
             (process-eval-when form env compile-time-too))
             (if compile-time-too
                 (internal-eval form env))
             (compile-form form env))
  (defun process-eval-when (form env compile-time-too)
      (let* ((situations  (cadr form))
             (body        (cddr form))
             (compile-p   (member 'compile situations))
             (load-p      (member 'load situations))
             (eval-p      (member 'eval situations)))
          (cond ((or (and compile-p load-p)
                     (and eval-p load-p compile-time-too))
                 (process-top-level-form `(progn ,@body) env t))
                 (process-top-level-form `(progn ,@body) env nil))
                ((or compile-p
                     (and eval-p compile-time-too))
                 (dolist (f body)
                     (internal-eval f env)))
  ;;;    do the obvious things.
  ;;; INTERNAL-EVAL evaluates "form" in lexical environment "env".

Cost to Users:

  Technically, none. Either proposal is technically upward compatible
  with CLtL.

  Proposal GENERALIZE-EVAL might force some extended implementations to
  change incompatibly. As such, some users who depend on 
  implementation-dependent extensions might have to adjust their code
  somewhat to deal with those changes.

  Proposal GENERALIZE-EVAL-NEW-KEYWORDS does not force implementations
  to change incompatibly, so has no forced impact on users.

Cost of Non-Adoption:

  EVAL-WHEN is a mess. Using it as the low-level substrate into which
  defining macros should expand, and guaranteeing any predictable effects
  of those macros in non-top-level situations is currently difficult and
  would continue to be so in the absence of some resolution on this issue.


  The costs of non-adoption would be avoided:  it would be possible to
  use EVAL-WHEN in many situations where it cannot currently be used

  The portability of many existing tools which use EVAL-WHEN internally
  in macros will be enhanced.


  This generalization of the meaning makes the purpose and uses of 
  EVAL-WHEN less mysterious. In that sense, aesthetics are simplified


  The cleanup issue LOCALLY-TOP-LEVEL would make LOCALLY also "pass
  through" top-level-ness to its body.  The reason why that is not 
  addressed in this issue is that it involves making LOCALLY a special

  Pitman and Moon don't care whether we say `top level,' `top-level,' or
  `toplevel.' The spelling choices in this writeup are arbitrary. If
  necessary, the proposal GENERALIZE-EVAL-NEW-KEYWORDS could be amended
  to propose :COMPILE-TOP-LEVEL, etc.

  Pitman, Moon, and Bob Laddaga (a Symbolics Cloe implementor) support
  both of these proposals.  Pitman and Laddaga have a preference for
  GENERALIZE-EVAL-NEW-KEYWORDS.  Moon is neutral about which should be

  Sandra Loosemore says:
    I still feel somewhat uncomfortable with the definition of EVAL-WHEN
    presented here, mostly because its nesting behavior is so unintuitive
    (as in test case number 6).  We have also had a hard time in deciding
    what the term "top-level" really means; the definition presented here
    is rather arbitrary.  However, since we have run out of time in which
    to come up with acceptable alternatives, I'm willing to go along with
    proposal GENERALIZE-EVAL.  It is compatible with the description in
    CLtL but presented in a more coherent way, and I think it is an
    improvement.  On the other hand, I don't really like the idea of
    changing the names of the keywords; if we are going to make an
    incompatible change, the right thing to do would be to throw out
    EVAL-WHEN entirely and start from scratch.