[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Issue: EVAL-WHEN-NON-TOP-LEVEL (Version 5)

Moon and I were not happy with the previous proposals on this issue, so
we wrote the following alternative proposal.  This proposal was not an
overnight project -- it's the result of quite a bit of careful thought,
discussion and review. I'm pretty happy with the outcome. It seems to
deal nicely with some issues that have bugged me for a while. I hope that
other people will like it, too.

Forum:        Compiler
References:   EVAL-WHEN (CLtL pp69-70),
Edit History: 06-May-88, Version 1 by Sandra Loosemore
              16-Dec-88, Version 2 by Loosemore (alternate direction)
              30-Dec-88, Version 3 by Loosemore (minor wording changes)
              07-Jan-89, Version 4 by Loosemore (update discussion)
              09-Feb-89, Version 5 by Pitman and Moon (some major changes)
Status:       For Internal Discussion

Problem Description:

  The current description of how the compiler should handle EVAL-WHEN
  only makes sense when it appears as a top-level form in the file being
  compiled. Is it legitimate for EVAL-WHEN to appear in non-top-level
  locations? Even if it is legitimate, what does it mean?
  Another issue, referred to here as ``the EVAL-WHEN shadowing problem,''
  is that some people have complained that shadowing the symbols EVAL,
  COMPILE, or LOAD means that you have to also either shadow EVAL-WHEN
  and define it to recognize the new symbol, or else you must resign
  yourself to writing (EVAL-WHEN (... LISP:EVAL ...) ...),etc. all over.
  While the goal here is not to solve this problem, it might be possible
  to solve both problems at once.


  The proposal which follows was constructed with the following goals
  in mind:

    1. The lexical and dynamic environment for the EVAL-WHEN body should
       be the same for each situation.  That is, the body should ``mean
       the same thing'' regardless of which situation is being processed.

    2. The evaluation context for EVAL-WHEN should be the current
       lexical environment.

    3. At execution time, EVAL-WHEN should always return the result of
       its last form if execution of the body occurred, or NIL if the
       body was not executed.

    4. If a top-level EVAL-WHEN has a LOAD keyword, its body should 
       inherit top-level-ness during normal processing. This permits the
       use of (EVAL-WHEN (EVAL COMPILE LOAD) ...) at top-level to mean
       simply "Do whatever would normally be done for this body, but
       also do something at compile time." This, in turn, will later be
       the key to allowing defining forms to be usefully described in
       terms of EVAL-WHEN.

    5. Non-top-level expressions should have no effect until they are
       executed. This is the key to making sure that any necessary
       environment is present. Since the COMPILE keyword forces effects
       to occur earlier than execution time, it follows from this that
       any correct solution must not allow the COMPILE keyword to have
       an effect at other than top-level.

  To accomplish these goals, we formulated the following model:

    The purpose of EVAL-WHEN is to accomodate the fact that some of the
    semantic processing of an expression may usefully be partitioned
    between compile time and run time in some circumstances.

    (EVAL-WHEN (EVAL) <code>)
    describes a general technique for accomplishing some particular goal
    at normal program execution time. However, the pair of expressions
    (EVAL-WHEN (COMPILE) <code-A>)
    (EVAL-WHEN (LOAD) <code-B>)
    can be used to describe an alternate technique for implementing part
    of the effect (A) at compile-time, and part of the effect (B) at


  Replace the description of EVAL-WHEN with the following:

  EVAL-WHEN ({situation}*) {form}*                      [Special Form]

  The body of an EVAL-WHEN form is processed as an implicit PROGN, but
  only in the situations listed.  Each SITUATION must be a symbol,
  either COMPILE, LOAD, or EVAL.

  The use of COMPILE and LOAD controls whether and when processing
  occurs for top-level forms. The use of EVAL controls whether
  processing occurs for non-top-level forms.

  The EVAL-WHEN construct may be more precisely understood in terms of
  a model of how the file compiler, COMPILE-FILE, processes forms in a
  file to be compiled. This model is totally general and is also used
  for other processors (such as the runtime compiler, COMPILE, and the
  evaluator, EVAL), but many of the cases which are necessary to explain
  the file compiler do not come up in other processors.

  Successive forms are read from the file by the file compiler using 
  READ. These top-level forms are normally processed in what we call
  `not-compile-time' mode. There is one other mode, called 
  `compile-time-too' mode, which can come into play for top-level
  forms. The EVAL-WHEN special-form is used to annotate a program
  in a way that allows the program doing the processing to select
  the appropriate mode.

  Processing of Common Lisp forms to be executed or compiled works
  as follows:

   If this is a top-level form in the file compiler,
   then if the form is an EVAL-WHEN,
        then if {both the COMPILE and LOAD situations are specified},
                or {{both the EVAL and LOAD situations are specified}
                    and {the mode is compile-time-too}},
          then process (PROGN . body) as a top-level form
               in compile-time-too mode,
          else if the LOAD situation is specified,
            then process (PROGN . body) as a top-level form
                 in not-compile-time mode,
            else if {the COMPILE situation is specified},
                    or {{the EVAL situation is specified}
                        and {the mode is compile-time-too}},
                 then evaluate the form (PROGN . body),
                 ;; {neither COMPILE nor LOAD}, and
                 ;; {not EVAL in compile-time-too mode}
                 else do nothing,
        ;; a top-level form in the file compiler that is not an EVAL-WHEN
        else if in compile-time-too-mode,
             then evaluate the form and then perform normal compiler
                  processing of the form as a top-level form,
             ;; not-compile-time mode
             else perform normal compiler processing of the form as a
                  top-level form,
   ;; not a top-level form in the file compiler: one of {in the interpreter},
   ;; or {in COMPILE}, or {in the file compiler not at top level}
   else if the form is an EVAL-WHEN,
        then if the EVAL situation is specified,
             then process (PROGN . body)
             else process NIL
        ;; not an EVAL-WHEN
        else process the form normally

  (PROGN . body) designates a form constructed by extracting the
  body forms from another form (EVAL-WHEN (situation...) . body).

  "Process" means to perform macroexpansion and the actions indicated by
  this decision tree.  COMPILE-FILE "processes" each form read from the
  file as a top-level form in not-compile-time mode (i.e., with
  compile-time-too mode off).  COMPILE and EVAL process each form
  they encounter as a normal form (i.e., a form that is not a top-level
  form); COMPILE-FILE does the same for non-top-level forms.
  "Evaluate" means to execute the form in the lexical environment in
  which the EVAL-WHEN appeared and in the compiler's executing dynamic

  "Normal compiler processing" means to process a top level form in
  the defined fashion for compilers, e.g. compiling function definitions.
  "Normal processing" means to execute a form if this is EVAL or
  to compile the form if this is COMPILE or COMPILE-FILE.


  The following effects are logical consequences of the above proposal:

   * It is never the case that the execution of a single EVAL-WHEN
     expression will execute the body code more than once.

   * The keyword `EVAL' is a misnomer because execution of
     the body need not be done by EVAL. In compiled code, such as
     it is permissible (even desirable) for the call to PRINT to be compiled.

   * Macros intended for use in top-level forms should arrange for all
     side-effects to be done by the forms in the macro expansion.
     The macro-expander itself should not do the side-effects.

       Wrong:  (defmacro foo ()
       Right:  (defmacro foo ()
                 `(eval-when (compile eval load) (really-foo)))

     Adherence to this convention will mean that such macros will behave
     intuitively when placed in non-top-level positions.

   * Placing a variable binding around an EVAL-WHEN reliably captures the
     binding because the `compile-time-too' mode cannot occur (because 
     introducing a variable binding would mean we were not at top level).
     For example,

        (LET ((X 3))

     will print 3 at execution [load] time, and will not print anything at
     compile time.  This is important so that expansions of DEFUN and 
     DEFMACRO can be done in terms of EVAL-WHEN and can correctly capture
     the lexical environment.

        (DEFUN BAR (X) (DEFUN FOO () (+ X 3)))

     might expand into

        (DEFUN BAR (X) 
                 (EVAL-WHEN (EVAL LOAD)
                   (SETF (SYMBOL-FUNCTION 'FOO) #'(LAMBDA () (+ X 3))))))

     which would be treated the same as

        (DEFUN BAR (X) 
          (SETF (SYMBOL-FUNCTION 'FOO) #'(LAMBDA () (+ X 3))))

     by the above rules.

   * The following table is an alternate way of describing the proposed actions:

     Level   COMPILE LOAD EVAL  C.t.t.   Action                 Level C.t.t.
     top       Yes   Yes  --     --      Process (PROGN body...)  top  Yes
     top       No    Yes  Yes    Yes     Process (PROGN body...)  top  Yes
     top       No    Yes  Yes    No      Process (PROGN body...)  top  No
     top       No    Yes  No     --      Process (PROGN body...)  top  No
     top       Yes   No   --     --      Evaluate (PROGN body...)
     top       No    No   Yes    Yes     Evaluate (PROGN body...)
     top       No    No   Yes    No      do nothing
     top       No    No   No     --      do nothing
     top      not an eval-when   Yes     Evaluate it and then
                                         perform normal compiler processing
     top      not an eval-when   No      Normal compiler processing
     normal    --    --   Yes    --      Process (PROGN body...)  normal
     normal    --    --   No     --      Process NIL              normal
     normal   not an eval-when   --      Normal processing

     "C.t.t." is an abbreviation for compile-time-too mode

     "--" means don't-care

Test Cases:

  ;; #1: The EVAL-WHEN in this case is not at top-level, so only the EVAL
  ;;     keyword is considered. At compile time, this has no effect.
  ;;     At load time (if the LET is at top level), or at execution time
  ;;     (if the LET is embedded in some other form which does not execute
  ;;     until later) this sets (SYMBOL-FUNCTION 'FOO1) to a function which
  ;;     returns 1.
  (LET ((X 1))
  ;; #2: If this expression occurs at the top-level of a file to be compiled,
  ;;     it has BOTH a compile time AND a load-time effect of setting
  ;;     (SYMBOL-FUNCTION 'FOO2) to a function which returns 2.
    (LET ((X 2))
        (SETF (SYMBOL-FUNCTION 'FOO2) #'(LAMBDA () X)))))
  ;; #3: If this expression occurs at the top-level of a file to be compiled,
  ;;     it has BOTH a compile time AND a load-time effect of setting the
  ;;     function cell of FOO3 to a function which returns 3.
  ;; #4: This always does nothing. It simply returns NIL.
      (PRINT 'FOO4)))
  ;; #5: If this form occurs at top-level of a file to be compiled, FOO5 is
  ;;     printed at compile time. If this form occurs in a non-top-level
  ;;     position, nothing is printed at compile time. Regardless of context,
  ;;     nothing is ever printed at load time or execution time.
      (PRINT 'FOO5)))
  This is compatible with any guarantees made by CLtL, and extends the
  behavior usefully to non-top-level situations.

  This gives a useful meaning to EVAL-WHEN that supports useful and
  predictable behavior if defining macros are used in a non-top-level


  As in GENERALIZE-EVAL, but rename the EVAL keyword to :EXECUTE,
  the COMPILE keyword to :COMPILE-TOPLEVEL, and LOAD keyword to 

  Deprecate the use of keywords EVAL, COMPILE, and LOAD to EVAL-WHEN.
  For compatibility, they are supported in EVAL-WHEN
  at top-level, but their meaning is not defined elsewhere.


   The fact that the situation keywords chosen are not the same as
   those now used means that the change can be added in a way that
   is truly upward compatible (not only with CLtL but with existing
   practice in implementations which have chosen to extend or `clarify'
   the definition given in CLtL) since the meaning of EVAL, COMPILE,
   and LOAD in non-top-level situations (which was never spelled
   out in CLtL) can legitimately differ from the meaning of these
   new keywords.

   Using other names and/or the keyword package for the names of
   situations solves the EVAL-WHEN shadowing problem.

   The name `execute' does not promote the confusion that the body of an
   EVAL-WHEN must be executed only in the evaluator. It also does not
   promote the confusion that the body of an EVAL-WHEN, regardless of when
   executed, must run interpreted.

   The names `compile-toplevel' and `load-toplevel' emphasize the fact
   that these cases are not interesting in non-top-level positions.

Current Practice:

  In Symbolics Genera, the interpreter permits EVAL-WHEN in non-top-level 
  positions in a way that is compatible with this proposal but both the
  COMPILE and COMPILE-FILE functions complain about EVAL-WHEN in a
  non-top-level position.

  IIM describes its EVAL-WHEN as 
   (defmacro eval-when (situations &body body &environment env)
     (if (not (compiler-environment-p env))
         (when (member 'eval situations) `(progn ,@body))
           (when (member 'compile situations)
             (if (compiler-at-top-level-p env)
                 (mapc #'eval body)
                 (warn "Top-level form encountered at non-top-level.")))
           (when (member 'load situations) `(progn ,@body)))))

Cost to Implementors:

  The actual change to EVAL-WHEN in both cases is probably fairly
  localized and straightforward to make in most or all implementations.

  The second-order costs of proposal GENERALIZE-EVAL will vary depending
  on whether existing implementations have extended the definition of
  EVAL-WHEN in incompatible ways. If an implementation has made such
  extensions, there may be user and system code which depends on them
  and the cost of converting that code may be non-trivial. There is
  presumably also documentation impact.

  Proposal GENERALIZE-EVAL-NEW-KEYWORDS avoids most or all of the 
  second-order costs of proposal GENERALIZE-EVAL.

Cost to Users:

  Technically, none. Either proposal is technically upward compatible
  with CLtL.

  Proposal GENERALIZE-EVAL might force some extended implementations to
  change incompatibly. As such, some users who depend on 
  implementation-dependent extensions might have to adjust their code
  somewhat to deal with those changes.

  Proposal GENERALIZE-EVAL-NEW-KEYWORDS does not force implementations
  to change incompatibly, so has no forced impact on users.

Cost of Non-Adoption:

  EVAL-WHEN is a mess. Using it as the low-level substrate into which
  defining macros should expand, and guaranteeing any predictable effects
  of those macros in non-top-level situations is currently difficult and
  would continue to be so in the absence of some resolution on this issue.


  The costs of non-adoption would be avoided:  it would be possible to
  use EVAL-WHEN in many situations where it cannot currently be used

  The portability of many existing tools which use EVAL-WHEN internally
  in macros will be enhanced.


  This generalization of the meaning makes the purpose and uses of 
  EVAL-WHEN less mysterious. In that sense, aesthetics are simplified


  Pitman and Moon don't care whether we say `top level,' `top-level,' or
  `toplevel.' The spelling choices in this writeup are arbitrary. If
  necessary, the proposal GENERALIZE-EVAL-NEW-KEYWORDS could be amended
  to propose :COMPILE-TOP-LEVEL, etc.

  Pitman, Moon, and Bob Laddaga (a Symbolics Cloe implementor) support
  both of these proposals.  Pitman and Laddaga have a preference for
  GENERALIZE-EVAL-NEW-KEYWORDS.  Moon is neutral about which should be