MAKE-LOAD-FORM can handle circularities [was Compilation implications]

[jrose@Sun.COM (John Rose)]

   Date: Sat, 7 Jan 89 01:04 EST
   From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>

       Date: Thu, 5 Jan 89  19:06:02 CST
       From: David N Gray <Gray@DSG.csc.ti.com>

       > Probably it would be a better idea for MAKE-LOAD-FORM to return two
       > values, where the first value is a form that will create the object and
       > the second value is a form that will further initialize the object?
       > This way the order of evaluation requirement is more explicit.  It's
       > upward compatible since the second value can be NIL if you don't need
       > it. 

       Yes, that sounds good except for the problem of how to pass the object to
       the second form.
    ...

   What I had in mind was ((LAMBDA (object) ...code...)).  But I like your
   suggestion of evaluating the form in an environment where * is bound to the
   object better.  Other people should check me on this, it might just be that
   you appealed to my sense of hackish kludgery.

There's a less kludgey way to get this taken care of:

Simply allow the second form to contain a reference to the object
being dumped.

Neat, yes?  File-level EQ preservation works to restore the embedded
object reference "for free".

And, while we're at it, give the programmer a break, and let the second
form be optional.

Here's a more complete description of these ideas:

Define a generic function MAKE-LOAD-FORM which takes one argument and
returns one or two values.  This function is called whenever
COMPILE-FILE needs to dump an object whose class is of type
STANDARD-CLASS or STRUCTURE-CLASS.  (Call these types "user defined".)
It returns one or two Lisp forms, which when passed at load time to EVAL
will construct an object which is, in some class-specific sense,
"equivalent" to the original object.

Call the first form returned by MAKE-LOAD-FORM the "allocator", and the
second form the "initializer".

The allocator must wholly or partially build the reconstructed object,
and return an ordinary Lisp reference to it.  The initializer, if
supplied and non-null, must finish any initialization required by the
object's class.  It is an error if the result of this second form is not
EQ to the result of the first.

Both the allocator and initializer are dumped to and restored from the
binary file by COMPILE-FILE, by the usual means.  It is expected that
they will consist of list structure, possibly with object of user-defined
type at the fringe.

The allocator must be dumpable without reference to the original object.
That is, in the process of dumping the original object, the dumper must
not be called upon to output the original object again until the allocator
has been completely dumped.

The initializer may contain references to arbitrary objects.  In
particular, it will typically contain a reference to the original
object.  Because all references to a given object in a compiled file
remain EQ across load, this can be reliably ensured simply by having
MAKE-LOAD-FORM return a reference to the original object embedded in
its second argument.

While the initializer form is being read in, the reconstructed object is
possibly in an uninitialized state, analogous to the state of an object
between the time its reference has been created by ALLOCATE-INSTANCE
and it has been processed fully by INITIALIZE-INSTANCE.  Implementors
must take care in manipulating objects referenced by allocator and
initializer forms, as they would in manipulating partially initialized
objects inside INITIALIZE-INSTANCE.

(Think of the allocator as creating a reference to a chunk of storage,
which stands in for the object until such time as the initializer can
really create it.  Meanwhile, the reference can be stored in other data
structure, and such stored references will become fully valid when the
object is finally initialized.)

Note that it is possible for uninitialized objects to appear in either
of the allocator or initializer forms, but when the loading process
completes, all initalizers will have been run.

A programmer of a certain class may elect to return a null initializer,
and perform all initialization in the allocator (which could then be a
simple call to MAKE-INSTANCE).  In this case, some circular data
structures involving that class will not be dumpable.  However, such
"simply dumped" classes may take part in circularities, as long as any
such a circularity includes at least one object which returns two values
for MAKE-LOAD-FORM, and whose allocator form breaks the circularity by
omitting all references to the next object in the circle.  Therefore,
not all classes need support the two-phase dumping protocol.

(Roughly speaking, in order to win, you need to have "enough" classes
with "small" allocator forms.)

Example:
	(defclass tree-with-parent () (parent children))
	(defmethod make-load-form ((x tree-with-parent))
	  (values
	    `(allocate-instance (class-of x) :children ',(slot-value x 'children))
	    `(initialize-instance ',x :parent ',(slot-value x 'parent))))