[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>: Issue: LOAD-TIME-EVAL (Version 5)]
- To: cl-cleanup@sail.stanford.edu
- Subject: [Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>: Issue: LOAD-TIME-EVAL (Version 5)]
- From: masinter.pa@Xerox.COM
- Date: 17 Sep 88 15:15 PDT
I think this is a moderately useful issue to be resolves if the compiler
committee hasn't done so. I'm not on cl-compiler so I don't know what, if any,
progress has been made on the numerous issues that were deferred to them. Was
this one of them?
----- Begin Forwarded Messages -----
Return-Path: <CL-Cleanup-mailer@SAIL.Stanford.EDU>
Redistributed: xerox-cl-cleanupâ??.pa
Received: from SAIL.Stanford.EDU by Xerox.COM ; 08 JUN 88 14:00:52 PDT
Received: from STONY-BROOK.SCRC.Symbolics.COM (SCRC-STONY-BROOK.ARPA) by
SAIL.Stanford.EDU with TCP; 8 Jun 88 13:56:15 PDT
Received: from PEWEE.SCRC.Symbolics.COM by STONY-BROOK.SCRC.Symbolics.COM via
INTERNET with SMTP id 416590; 8 Jun 88 16:55:29 EDT
Date: Wed, 8 Jun 88 16:54 EDT
From: Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: LOAD-TIME-EVAL (Version 5)
To: CL-Cleanup@SAIL.STANFORD.EDU
cc: Moon@STONY-BROOK.SCRC.Symbolics.COM, KMP@STONY-BROOK.SCRC.Symbolics.COM
Message-ID: <880608165454.0.KMP@PEWEE.SCRC.Symbolics.COM>
Well, this really needs a lot more polish, but I don't have more time to
spend on it today and I wanted to get it out for people to look at
before we meet next week. This is definitely a rough cut and I'm sure it
won't want to go to committee in this form, but at least now all the
proposals are under a single cover and we don't have to play leap-frog
with Moon removing my proposal or my removing his (unless one of us
gives in).
-----
Issue: LOAD-TIME-EVAL
References: #, (p. 356), (EVAL-WHEN (LOAD) ...) (p. 69-70)
Category: ADDITION
Edit history: 06-Jun-87, Version 1 by James Kempf
17-Jul-87, Version 2 by James Kempf
12-Nov-87, Version 3 by Pitman (alternate direction)
01-Feb-88, Version 4 by Moon
(from version 2 w/ edits suggested by Masinter)
06-Jun-88, Version 5 by Pitman
(fairly major overhaul, merging versions 3 and 4)
Status: For internal discussion
Problem description:
Common Lisp provides reader syntax (#,) which allows the programmer
to designate that a particular expression within a program is to be
evaluated early (at load time) but to later be treated as a constant.
Unfortunately, no access to this capability is available to programs
which construct other programs without going through the reader.
Some computations can be deferred until load time by use of EVAL-WHEN,
but since EVAL-WHEN must occur only at toplevel, and since the nesting
behavior of EVAL-WHEN is quite unintuitive, EVAL-WHEN is not a general
solution to the problem of load-time computation of program constants.
Also, CLtL is vague about whether the result of this early evaluation
is re-evaluated at runtime. The meaning of #,exp in an for-evaluation
position is unclear. Although CLtL doesn't come out and say so explicitly,
portable code must currently use only '#,exp to get consistent behavior
across implementations.
CLtL is also vague on whether the result of a #, expression may be
treated as a read-only constant by the loader (and hence shared with
other programs). Users probably want some both read-only and modifiable
load-time constants, so this may not be simply an issue of deciding on
a single "right answer".
Proposal (LOAD-TIME-EVAL:QUOTED-MAGIC-TOKEN):
Add a function MAKE-LOAD-TIME-CONSTANT, as described here:
MAKE-LOAD-TIME-CONSTANT form env &optional read-only-p [Function]
FORM is a Lisp form. ENV is an environment of the sort received
by the &ENVIRONMENT argument to a macro.
When MAKE-LOAD-TIME-CONSTANT is called from the interpreter or the
COMPILE function, it simply evaluates FORM in the null lexical
environment and returns its value. When MAKE-LOAD-TIME-CONSTANT is
called during a file compilation, the result is a special object
that is recognized at load time, when it occurs inside a constant.
At load time, FORM is evaluated and its value is substituted for
the object.
MAKE-LOAD-TIME-CONSTANT uses its ENV argument and/or dynamic state
to determine whether it is being called during a file compilation.
Until Common Lisp is modified to specify the semantics of file
compilation more precisely, this is necessarily implementation
dependent.
The READ-ONLY-P argument designates whether the result can be considered
read-only constant. If NIL, the result must be considered ordinary,
modifiable data. If T, the result is a read-only quantity which may, as
appropriate, be copied into read-only space and/or shared with other
programs.
Specify that '(... #,exp ...) is equivalent to
#.`(... ,(MAKE-LOAD-TIME-CONSTANT exp NIL T) ...).
Rationale:
This approach is the most compatible with existing practice.
Cost to Implementors:
The cost to implementors will depend on how #, is implemented.
In some implementations, the primitives for implementing
MAKE-LOAD-TIME-CONSTANT may already exist, in others, more substantial
changes may be required.
Cost to Users:
This change is upward compatible with user code.
Benefits:
It would be possible for macros to expand into load time constants.
Examples:
Case QUOTED-MAGIC-TOKEN-1:
(defmacro print-software-version (&environment env)
`(quote ,(make-load-time-constant
'(format T "~A~%" (software-version))
env)))
When interpreted or processed during invocation of COMPILE, this
macro prints the value of (software-version) at macro expansion
time and expands into (quote nil). When macroexpanded during a
file compilation, printing is deferred until the compiled file is
loaded, and the constant is still (quote nil).
Case QUOTED-MAGIC-TOKEN-2:
(defmacro table-of-tables (&rest predicates &environment env)
`(quote ,(mapcar #'(lambda (predicate)
`(,predicate
,(make-load-time-constant
`(make-hash-table :test ',predicate)
env)))
predicates)))
(table-of-tables eql equal) expands into
(quote ((eql #<table :test eql>) (equal #<table :test equal>)))
except that when macroexpanded during a file compilation,
the tables are not created until load time. This example
shows that the <object> returned by make-load-time-constant is
recognized even when it is interior to a quoted constant.
Proposal (LOAD-TIME-EVAL:NEW-SPECIAL-FORM):
Add a new special form, LOAD-TIME-CONSTANT, which has the following
contract:
LOAD-TIME-CONSTANT form &optional read-only-p [Special Form]
All processing of the FORM is deferred until the expression is
in the "runtime" environment. Once that environment is available,
FORM is evaluated in the null lexical environment and the result
is both returned and saved for immediate access by the program
on subsequent evaluations.
In the interpreter, the FORM may be evaluated during pre-processing
(if any) or dynamically when the LOAD-TIME-CONSTANT is first seen
(in a non-pre-processing implementation). If the same LOAD-TIME-CONSTANT
expression is later seen again by the interpreter, the previously
obtained result is immediately retrieved and returned as the result
of evaluating the object; no re-evaluation occurs.
If the LOAD-TIME-CONSTANT expression is seen by the file compiler
(eg, COMPILE-FILE), the compiler arranges for all semantic processing
of FORM (including macro expansion) to occur at load time in a null
lexical environment (independent of whether any value has been cached
for interpreter use). At runtime, the result of that evaluation will
be treated as an immediate quantity; no re-evaluation occurs.
If a LOAD-TIME-CONSTANT expression is seen by the runtime compiler
(eg, COMPILE), the compiler checks for a cached value which may have
been produced by the interpreter. If one is found, it is used. If no
such value is found, the runtime compiler will evaluate the FORM in
a null lexical environment and use that value. The value used will be
treated as an immediate quantity in the code which is produced; no
re-evaluation occurs.
Note that since some implementations are compiled-only (that is, they
implement their interpreter using a compiler pre-pass) and some are
interpreted-only (that is, they implement their compiler as a null
operation and use only an interpreter), the question of whether the
interpreter or the compiler will end up doing the processing is left
somewhat vague. The programmer may assume only that the given FORM
will be evaluated only once for each time it is loaded into a runtime
environment.
Note, however, that in the case of quoted code (processed by explicit
use of EVAL), each call to EVAL is treated like a load. Caching may not
be implemented by having LOAD-TIME-CONSTANT displace its source level
call. So while
(DEFVAR *FOO* 1)
(DEFUN FOO () (LOAD-TIME-CONSTANT (INCF *FOO*)))
will increment *FOO* only once,
(DEFUN FOO () (EVAL '(LOAD-TIME-CONSTANT (INCF *FOO*))))
will increment *FOO* once each time FOO is called.
The READ-ONLY-P argument designates whether the result can be considered
read-only constant. If NIL, the result must be considered ordinary,
modifiable data. If T, the result is a read-only quantity which may, as
appropriate, be copied into read-only space and/or shared with other
programs. (Because this is a special form, this argument is -not- evaluated
and only the literal symbols T and NIL are permitted.)
Make #,exp be equivalent to (LOAD-TIME-CONSTANT exp T). As such, it
would -always- appear in a for-evaluation position, and never inside quoted
structure.
Rationale:
By making the description of LOAD-TIME-CONSTANT defined as a special
form, we eliminate the need for it to take an environment argument.
By making #, agree with LOAD-TIME-CONSTANT in terms of where it may be
used, we simplify the description of the resulting language.
As discussed in more detail elsewhere in this proposal, the #, syntax
is currently only reliably useful -inside- quoted structure, but this
is unnecessarily complicated for most known uses. Since this proposal
suggests a meaning for #, only -outside- quoted structure, it is an
incompatible change (though not one that would necessarily require
vendors to immediately remove support for existing code).
Cost to Implementors:
This is an incompatible change to the program interface.
The cost is not trivial, but is not particularly high.
Most of the "hard" substrate required to support this proposal
probably already exist; in most cases, what needs to change is
only the way in which the substrate is presented to the
programmer.
Some code-walkers would have to be taught about this new
special form. Such changes would likely be trivial.
Cost to Users:
Very few users probably use #, right now. A very small amount of code
might need to be rewritten (and recompiled).
The following kinds rewrites exemplify the total amount of work needed
in the few situations that use #, currently:
'#,exp => #,exp
'(... #,exp ...) => #,`(... ,exp ...)
`(... #,exp ...) => `(... ,#,exp ...)
Some user-defined code-walkers would have to be taught about
this new special form. Such changes would also be likely be trivial.
Although change to #, is an incompatible one, vendors would be free to
provide compatibility support for the old behavior for whatever period
they deemed appropriate. #, is equivalent to, but not the same as,
(LOAD-TIME-CONSTANT exp T), so it might in fact expand into
(SYSTEM::LOAD-TIME-CONSTANT exp T) which might be both a macro that
expanded into (LOAD-TIME-CONSTANT exp T). SYSTEM::LOAD-TIME-CONSTANT
might be a magic token which was treated specially within quoted
structure as an unportable extension during a transition period.
Benefits:
Relatively consistent interpreter/compiler treatment of this special form
would be possible.
Pretty printing expressions in which LOAD-TIME-CONSTANT occurred would be
possible.
An expression such as `(... #,(... , ...) ...) would be meaningful.
Manipulating expressions involving uses of LOAD-TIME-CONSTANT would be
possible. Currently, calling READ is enough to trigger resolution of the
constant, so a program doing file-to-file source code rewrites must
either use a special readtable or resign itself to premature resolution
of load time constants.
Expressions involving quoted LOAD-TIME-CONSTANT expressions would be
possible to quote. Currently, the #, feature is syntactic, not semantic,
and so is not sensitive to levels of quotation. You can refer to a
load time constant by writing '#,exp but you cannot refer in turn to that
expression which refers to a load time constant by writing ''#,exp .
Under this NEW-SPECIAL-FORM proposal #,exp and '#,exp and ''#,exp (and
so on) are all usefully distinct.
It would be possible to use , in a #, expression. For example, the
expression:
Examples:
Case NEW-SPECIAL-FORM-1:
(defmacro print-load-timestamp ()
`(print (load-time-constant
`(load-timestamp ,(incf *foo*) ,(get-universal-time))
t)))
(defvar *foo* 0)
(defun test-1 () (print-load-timestamp))
(test-1)
CLtL does not define this situation.
Under this proposal, this code would print
(LOAD-TIMESTAMP 1 <<a-universal-time>>)
at the time the test case is loaded, whether interpreted or compiled.
Subsequent calls to (TEST-1) should print the identical expression.
Currently, no known implementation supports the proposed behavior.
Case NEW-SPECIAL-FORM-2:
(defun test-2 () (print #,'(+ 3 4)))
CLtL does not adequately define this situation.
Under this proposal, this would print (+ 3 4), whether interpreted
or compiled.
Currently, some compilers complain about the syntax, some arrange for
it to print (+ 3 4), and some arrange forit to print the result of
(+ 3 4), or 7.
Case NEW-SPECIAL-FORM-3:
(defun test-3 () (print '#,'(+ 3 4)))
Under CLtL, this would print (+ 3 4).
Under this proposal, the behavior would be undefined.
Currently, most implementations support the proposed behavior.
Case NEW-SPECIAL-FORM-4:
(pprint '(+ x #,(sqrt x)))
prints something re-readable like
(+ X #,(SQRT 2)) or (+ X (LOAD-TIME-CONSTANT (SQRT 2) T))
but not something like
(+ X 1.4142135)
Currently, no implementation is known to already support the
proposed behavior, but in principle it is possible for a valid
implementation to already do so.
Case NEW-SPECIAL-FORM-5:
(defmacro foo (x y)
`(member ,x #,`(foo ,,y baz)))
(macroexpand '(foo *bar*)) => ??
Under CLtL, this situation is not adequately defined.
Under this proposal, the macroexpansion would be
(member foo #,`(foo ,*bar* baz))
Currently, this triggers a read error such as "comma not in backquote"
or "SYSTEM::BACKQUOTE-COMMA undefined function" in most
implementations.
Current practice:
Although most implementations provide a substrate which would allow
program-mediated access to load time evaluation in some way, the language
only defines access to this substrate through the sharpsign read syntax.
Costs of Non-Adoption:
There are numerous possible uses for this facility. Among them are:
* Version control and system building facilities.
* The Common Lisp Object System.
* Language translators which desire to emulate "linking".
While every implementation of Common Lisp could certainly provide an
implementation specific facility capable of supporting such facilities,
portability of such facilities would suffer.
Benefits:
Portability and extended language power. The nature of each proposed
extension is such as to enable other extensions to be added more
gracefully. The Common Lisp Object System is a clear example.
Aesthetics:
These proposals fill a hole in the spectrum of alternatives for
deferring evaluation until a compiled file is loaded. Currently, code
which is read by the reader can arrange for it to be done, as can
top level code, but embedded code cannot. As such, these proposals
clarify and regularize existing parts of the language. Also, by
adding program-accessible entry points to facilities already provided
in a more contrived way, it makes the language easier to use.
Discussion:
There is likely to be some controversy about this proposal, since
there is no universally agreed upon formal processing model for
Common Lisp.
The cleanup committee seems to generally approve of the idea of a
load-time-eval capability, but a number of the details seem to need
ironing out.
Moon supported a previous draft of QUOTED-MAGIC-TOKEN. In this draft,
KMP changed the presentation and also added the READ-ONLY-P argument
in order to make it sit nicely with the alternate proposal,
NEW-SPECIAL-FORM. It's more than slightly possible that after all this
editing, Moon will have some problems with this version and want to
submit a refined draft.
Pitman supports NEW-SPECIAL-FORM.
Rees has expressed strong support for the idea of implementing #, as a
new special form rather than perpetuating the current state of affairs.
He had some input into the high-level content of this proposal, though
he hasn't reviewed any drafts. This paragraph is intended primarily to
incite him to say something definitive one way or the other.
----- End Forwarded Messages -----