top-level, eval-when: it's not too late

[Rob.MacLachlan@WB1.CS.CMU.EDU]

Before we go off the deep end arguing about what top-level is and how forms
should differ at top-level and elsewhere, we should seriously consider
top-level-less alternatives.  I believe that the concept of top-level is
totally unnecessary for a well defined Common Lisp semantics.  Furthermore,
I believe the concept of top-level is a blight on Lisp semantics:
 -- Differing top-level semantics creates a totally gratuitous context
    dependency.  (can you say "referential transparency"?)
 -- It is complex to define what top-level means (and difficult to
    understand complex definitions).
 -- It is difficult (perhaps impossible) to come up with a definition for
    top-level that allows as much power as when there is no concept of
    top-level.


What does top-level mean?

Moon:
    I'd like to point out that the defining characteristic of top-level is
    not that the lexical environment is null, but that the lexical environment
    does not contain any variables, functions, blocks, or go tags; it
    contains only declarations, macros, and the COMPILE-FILE remote/local
    environment business.  Also top-level is not nested inside any flow
    of control (conditionals, iterations, catches [which are essentially
    a kind of conditional]); that's why the compiler can treat defmacro,
    for example, specially at top level, because it knows that the defmacro
    will be executed exactly once at load time and in a fully known lexical
    environment.

In other words, a top-level form:
 -- can be proven to be evaluated at load time, and
 -- has no run-time values in its environment, and thus can be evaluated at
    compile time.

But why do we need this?  The goal of this special treatment of top-level
forms seems to be to allow the user to believe in a naive model: there
aren't any special rules form compile-time evaluation.  The compiler only
"dares" to evaluate forms at compile time when it can prove that the same
evaluations would happen in the same order at run time.

The assumption is that it is that discretion is the the better part of
valor: it is better not to evaluate a form at compile time when the naive
model might be violated.  Underlying this is an assumption that:
 -- It is very rare for it to be desirable to compile-time evaluate a form
    and for the compiler to be unable to prove that the form will be
    evaluated at run-time, and/or
 -- It is "relatively harmless" to omit compile-time evaluation when
    evaluation might cause confusion.

I don't agree with either of these assumptions.  These assumptions only
hold for a restricted Common Lisp programming style.  Given extensive
top-level use of FLET, LET, etc, many defining forms end up in an
environment that is clearly not "top-level" as defined above.  Yet is is
*not* harmless for the compiler to forget to notice variable and function
declarations and definitions.  Failure to notice a function definition
results in confusing spurious warnnings.  Failure to notice a DEFVAR
results in code that just doesn't work.

The alternative is to not be conservative about doing compile-time
evaluation: to always evaluate an eval-when (compile) wherever it appears.
In this scheme, compile time evaluation is clearly a visibly separate
process from load-time evaluation: forms could be evaluated at compile time
that would not be evaluated at load time.  There is potential for confusion
here, but in reality it isn't very confusing.

The key difference lies in the difference between what the compiler can
easily prove and what is true.  Even when buried inside all kinds of gunk,
defining forms will usually be evaluated at load time even though the
compiler can't prove it.  The same naive model holds as for the top-level
system.  The difference is in where you fall off the edge of the naive
world.  In the top-level-only evaluation model, the user finds he just
can't do what he wants to do, whereas in the always-evaluate model, he
finds that he has to insert some form that inhibits compile-time
evaluation.


Here is my proposal:

Take the original EVAL-WHEN-NON-TOP-LEVEL proposal as the meaning of
EVAL-WHEN everywhere.  Continue to define compiler "noticing" to take place
as though through EVAL-WHEN.  That's all you need.

A minor wart is that the compiler currently needs to notice some "magic
functions": package functions, provide & require, proclaim.  At least some
of these functions have reasonable run-time uses in addition to the magic
compile-time uses, so it could cause some confusion to always evaluate
these forms at compile time wherever they appear.  Fortunately,
the cleanup process seems to be eliminating most of these semantically
bletcherous functions.

Even if the magic functions aren't all eliminated, we still don't need
top-level.  As long as there is some eval-when-oid mechanism that inhibits
enclosed eval-when (compile)s, any uses of magic functions that don't want
to be magic can be wrapped in this form (LATE-EVAL-ONLY in my original
compiler cleanup proposal.)

  Rob