Load time/compile time

[Jonathan Rees <Rees at YALE>]

    Date: 3-Dec-82 11:28PM-EST (Fri)
    From: Drew McDermott <Mcdermott>
    To:   T-Discussion

    I have been wrestling with the T feature that all macros are expanded
    when a function is defined. ...

This is a real problem and I sympathize.  But I'm not sure just what it
is that you're doing that really requires the use of macros; why won't
procedure integration do what you want?  I presume that : is a readmacro;
what does it expand into?  Are you saying that most of the work is done
at macro-expansion time, and little is done at run-time?

I guess what I'm saying is that the more procedural (DEFINE) and less
syntactic (DEFINE-MACRO) you make your code, the better off you are; you
are supposed to be able to depend on a smart compiler to make this
supposedly less efficient way of doing things work well.  If it's too
inefficient in INTERPRETED code, then perhaps the interpreter should be
made smarter.

Is it the case that you have macros which expand into BLOCKs which
define both macros and procedures?  This is an awkward (and some might say
unclean) situation because it mixes the two semantic levels.

    On the other hand, there is really no reason for things to stay
    the way they are.  As far as I can see, adding a hack to the
    interpreter to delay macro-expansion until the first call would
    solve the problem.  (Sorry -- if hacks offend you, think of it
    as a kind of functional call by need.)

I don't see how this can be done in general for compiled code.  Sure,
one could make the compiler do two passes over the file, but then
we get still further away from some nice properties (like
 (COMPILE (APPEND A B)) <=> (APPEND (COMPILE A) (COMPILE B)), so to
speak) that we would like the language to have; what the hell is a
"file," anyhow?  Why make files be part of the semantics?

    If all that seems esoteric, here is a practical consideration
    that everyone will appreciate:  One often loads a file full of
    interpreted functions with the intention of running only a
    fraction of them.  Why should you have to pay for macro-expanding
    all of them?

This is completely reasonable; in fact I have been considering putting a
delaying tactic into the evaluator for quite some time now, and will get
around to it one day.  A two-pass compiler would be a wonderful thing
also, because then all sorts of optimizations are possible, not just
retroactive DEFINE-INTEGRABLE's.  But I certainly don't want to have
anything like this in the language DEFINITION because it then becomes
much harder to write semantically correct evaluators (that is, define
the language's semantics).  I think we've been careful so far not to
"overdefine" the language; its semantics are close to being tractable.
This is T's (and Scheme's) main advantage over other Lisp dialects, and
one must be vigilant at every step to avoid throwing that advantage
away.

My opinion isn't cast in concrete; there's a lot in what you say, and it
might be basically the right way to go.  But it bothers me too.  The way
in which the system deals with macros internally is going to change a
great deal in the not-too-distant future (Sussman and a few of us other
Schemers are working on the semantics of macros), perhaps in such a way
as to shed light on, or even force, this issue.

---------
I'm sure you realize that SET is functional rather than syntactic in the
sense that, e.g., (LET ((X CAR)) (SET (X Y) FOO)) sets the CAR of Y.
Thus also if you compile (SET (BAZ Y) FOO) the compiler does NOT need
access to BAZ's definition.  This is just because (SET (BAZ Y) FOO) is
equivalent to ((SETTER BAZ) Y FOO), and SETTER is a procedure (which the
compiler knows a little about, the same way it knows about CAR).  If BAZ
isn't available then you just lose some possible optimizations, the same
way as if you're missing the definition of an integrable procedure.

---------
and by the way, I use the term "load-time" to refer to what you call
"read-time" because it really doesn't have anything to do with what READ
does.