gc-by-area and host uptimes

[Jeffrey Mark Siskind <Qobi@ZERMATT.LCS.MIT.EDU>]

    I dont recall the reason why you were so reluctant to cold-boot --- I
    guess it is much more than merely avoiding re-initing the state of the
    machine. It seems you are trying to drive the beast beyond its practical
    limits, and the existence of :Start GC :Immediately By-Area is your
    saviour. Asking for more is reasonable, but thats all you can do, for
    such an unusual need (which, dont you admit, is the case?)

I am reluctant to cold-boot because it takes me over 24 hours to restore
the machine to the state it was in prior to cold-boot. This is because of
two reasons. First, I am running SPIRE and have lots of cached info like
spectrograms and such stored in virtual memory. Second, my own Prolog
compiler generates Lisp from Prolog with the property that it takes
several hours for the Symbolics lisp compiler to compile the code
generated. (It is not that my code is so outrageous, it is just that
the symbolics compiler is tailored to hand-written functions which tend
to be small. Anybody who has tried to use the symbolics compiler on
machine generated code has found that the compiler blows up on functions
that are longer than a few pages in length --- even if such code is
very straighforward and Pascal-like. I don't think that it is unreasonable
to use Lisp as a target language for another compiler and expect that the
Lisp compiler can handle large machine-generated functions.) Anyway, so
for my use, a mark-sweep-gc would be much faster and convenient than
reboot, since I also must hand-hold my machine during reboot to restore
the desired state of my machine.

I don't think that I am "trying to drive the beast beyond its practical
limits". All I am doing is running task that create a lot of temporary
garbage which outlives the EGC. But since I can't afford to set aside
half of my memory for a dynamic GC, and don't want to hand-hold a
gc-by-area, I really could use a mark-sweep GC. Actually, a mark-sweep
GC is a very low-tech item compared with EGC. Practically, every
Lisp implementation in the world short of Symbolics has a mark-sweep GC.
I don't think that my need is so unusual. It should be encountered by
any application which generates temporary garbage which outlives the
EGC and who can't afford the other two aforementioned solutions.
I know of a least one other drastically different application which
I worked on (VLSI design automation) which although very different
from Prolog and Speech Recognition, also had a strong need for mark-sweep
GC to run long compute intension jobs unattended. To this day, symbolics
has yet to address the issue of running long consing-intensive jobs
unattended.

        That doesn't bother me. Several gc-by-areas takes me also twelve hours. During
        that time I am dead affraid to touch my machine for fear that I might cons
        and destroy the saftey predictions of the GC mechanism. I'd rather be able to do
        that unattendend over night or during the weekend. For all I care a GC that
        took 72 hours that I ran once every two or three months wouldn't be that bad
        at all.

    Sure, I agree with this of course. Do you know how you would implement
    what you are asking for? What time would it take? Do you have the time,
    if Symbolics does not?

Storage management (i.e. paging, gc, the scheduler) are among the lowest level,
most internal components of a Lisp system. The techniques needed to modify
or enhance such components are not documented. I would expect them to make use
of information which contains highly guarded corporate secrets. In general,
the technology behind mark-sweep GC is well known. Anyone who has taken an into
data-structures course as an undergraduate should know how to do it. The problems
are its interaction with the rest of the system. I don't think it would take
a tremendous amount of time. After all, Symbolics did implement one in the
process of building Ivory precisely because it was simpler than the other GC
methods and was a temporary measure to get the system up and running. If
someone inside Symbolics would tell me all of the necessary details, I
might consider trying to do it myself.

        Not me, I am still asking for it, based on the fact that I beleive that it
        is a good idea irrespective of its implementation status.

    Asking is fine, and I know you realize the burden placed on the
    development staff for 1all0 their demands.

        I hope that it is not the final comment on the matter.

    Well, I suggest that it is for the moment, given all the other things
    teh developers are working on, with much larger impact and wider demand
    across the product range. 

I don't necessarily beleive that Symbolics (or any company for that matter)
always has the insight to work on only the things with the widest impact.
Sometimes companies misjudge the relative impact of different alternatives
and sometimes they just plain ignore the impact issues and work on things
based upon their whims. Now some things that Symbolics is working on
clearly do have far greater impact than mark-sweep-gc. Like lower-cost
higher-performance hardware like Ivory. On the other hand, there have
been some significant development efforts by Symbolics, ones which
I assume take more effort than mark-sweep-gc, which I think have far
less impact than it. For example: the Fortran and Pascal compilers,
Joshua, Concordia, the document examiner, Statice, most of the
features of dynamic windows besides infinite scrolling, ...
        Jeff