MAKE-LOAD-FORM can handle circularities [was Compilation implications]

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

    Date: Tue, 10 Jan 89 19:03:49 PST
    From: jrose@Sun.COM (John Rose)

    Simply allow the second form to contain a reference to the object
    being dumped.

    Neat, yes?  File-level EQ preservation works to restore the embedded
    object reference "for free".

Right.  You are so smart!

    And, while we're at it, give the programmer a break, and let the second
    form be optional.

That was always the intention.  Complex capabilities shouldn't make doing
simple things harder.

    Here's a more complete description of these ideas:

I pretty much agree with your description.  I still hope to find time
to write a second version of the proposal, which will incorporate what
you said, except that I will try to write it in less implementation
oriented terms and more language oriented terms.

    Define a generic function MAKE-LOAD-FORM which takes one argument and
    returns one or two values.  This function is called whenever
    COMPILE-FILE needs to dump an object whose class is of type
    STANDARD-CLASS or STRUCTURE-CLASS.  (Call these types "user defined".)
    It returns one or two Lisp forms, which when passed at load time to EVAL
    will construct an object which is, in some class-specific sense,
    "equivalent" to the original object.

    Call the first form returned by MAKE-LOAD-FORM the "allocator", and the
    second form the "initializer".

    The allocator must wholly or partially build the reconstructed object,
    and return an ordinary Lisp reference to it.  The initializer, if
    supplied and non-null, must finish any initialization required by the
    object's class.  It is an error if the result of this second form is not
    EQ to the result of the first.

If you remove this seemingly useless error check, you don't have to special
case NIL as a second value.  (EVAL NIL) never hurts.

    Both the allocator and initializer are dumped to and restored from the
    binary file by COMPILE-FILE, by the usual means.  It is expected that
    they will consist of list structure, possibly with object of user-defined
    type at the fringe.

    The allocator must be dumpable without reference to the original object.
    That is, in the process of dumping the original object, the dumper must
    not be called upon to output the original object again until the allocator
    has been completely dumped.

    The initializer may contain references to arbitrary objects.  In
    particular, it will typically contain a reference to the original
    object.  Because all references to a given object in a compiled file
    remain EQ across load, this can be reliably ensured simply by having
    MAKE-LOAD-FORM return a reference to the original object embedded in
    its second argument.

    While the initializer form is being read in, the reconstructed object is
    possibly in an uninitialized state, analogous to the state of an object
    between the time its reference has been created by ALLOCATE-INSTANCE
    and it has been processed fully by INITIALIZE-INSTANCE.  Implementors
    must take care in manipulating objects referenced by allocator and
    initializer forms, as they would in manipulating partially initialized
    objects inside INITIALIZE-INSTANCE.

    (Think of the allocator as creating a reference to a chunk of storage,
    which stands in for the object until such time as the initializer can
    really create it.  Meanwhile, the reference can be stored in other data
    structure, and such stored references will become fully valid when the
    object is finally initialized.)

    Note that it is possible for uninitialized objects to appear in either
    of the allocator or initializer forms, but when the loading process
    completes, all initalizers will have been run.

    A programmer of a certain class may elect to return a null initializer,
    and perform all initialization in the allocator (which could then be a
    simple call to MAKE-INSTANCE).  In this case, some circular data
    structures involving that class will not be dumpable.  However, such
    "simply dumped" classes may take part in circularities, as long as any
    such a circularity includes at least one object which returns two values
    for MAKE-LOAD-FORM, and whose allocator form breaks the circularity by
    omitting all references to the next object in the circle.  Therefore,
    not all classes need support the two-phase dumping protocol.

    (Roughly speaking, in order to win, you need to have "enough" classes
    with "small" allocator forms.)

    Example:
	    (defclass tree-with-parent () (parent children))
	    (defmethod make-load-form ((x tree-with-parent))
	      (values
		`(allocate-instance (class-of x) :children ',(slot-value x 'children))
		`(initialize-instance ',x :parent ',(slot-value x 'parent))))

The example needs some debugging.