DEFSYSTEM

[Charles Rich <RICH at MIT-AI>]

In System 78.42, ZMail 38.5, microcode 836, on Lisp Machine Twenty-two:

Gentlemen -
  I have recently started using DEFSYSTEM in a fairly
sophisticated way and would like to summarize some thoughts and
suggestions.  My first general comment is that I have mixed
feelings about this utility.  I had a very difficult time
learning how to use it, which for an experienced user such as
myself should raise a warning flag.  I am not sure how much of
this difficulty is a result of the inherent complexity of the
application and how much is due to the poor design of the
utility.  I think there are definitely components of both.  I
think the basic idea of having an external declarative data
structure which expresses dependencies between files is correct.
We have clearly gotten past the point where we can just put
conditionalized top-level forms in files to make the right
things happen at the right time, as we used to in Maclisp --
there are just too many variations in how a large system can
be incrementally modified.
  Now for some specifics:

(1) An important distinction between complex tranformations
[defined by (defmacro (... si:defsystem-macro)) ] and simple
transformations [defined by si:define-simple-transformations]
is not made clear in the documentation, namely that complex
transformations cannot be used as dependencies.  I discovered
this by trying to use :DO-COMPONENTS as a dependency
for compilation of a module of the current system.  By looking
at the source code I found SI:DO-COMPONENTS-INTERNAL was
the corresponding simple transformation.

(2) The documentation of :DO-COMPONENTS could be improved.  The
manual actually says :DO-COMPONENTS is a simple transformation,
which is wrong.  Also, the documentation for :DO-COMPONENTS
just implies that this is a place to hang dependencies for the
components.  It didn't occur to me until I was told that putting
this special transformation in would cause the components to be
done right then rather than as the last transformation by
default.

(3) :COMPILE-LOAD-INIT is a real win for getting the world
recompiled when you change your macros.  However, there is
a problem when you change something in the macro file
which doesn't imply the world needs recompilation (e.g.
a comment or adding a new macro.)  Right now the only way
to hack this is to clobber the file date of the macro
qfasl file backward to before any of the dependent modules.
I suggest adding a keyword to MAKE-SYSTEM called ':TOUCH
(DCB told me UNIX has a similar feature of MAKE) which
sets a flag which controls a conditional in the definition of the 
the :COMPILE transformation:  when the flag is set, the
:COMPILE transformation only resets the file date of the qfasl,
rather than actually calling QC-FILE.  Then you could, for
example, do (MAKE-SYSTEM 'FOO ':COMPILE ':TOUCH) which would
be quite fast and more logical then back-setting the filedate
of the macro file.

(4) Once we are taking all the (EVAL-WHEN (LOAD COMPILE) (LOAD "foo macros"))
type stuff out of source files and putting it in a centralized
defsystem structure, I think it would make sense to take out
other such toplevel forms which rather than loading files, call certain
environment initialization functions.  I really like the idea
of having only source code in source files (and some
DEFVARs and DEFCONST initializations).  How about defining a
simple psuedo-transformation called :EVAL which takes
a list of forms to evaluate.  I succeeded in defining :EVAL
as a complex transformation, but then I couldn't use it in
the dependencies list, and the user defined code for defining
simple transformations really insists that they take module
specs as arguments.

  Well, that's all for now.  Thanks for your audience.  One
last question:  what are your plans for Defsystem?  Is it
going to stick around in its present form?  Are you amenable
to adding features as above?  How high up on the list of priorities
is it?
				-Chuck.