Optimize World command

["David L. Andre" <DLA@diamond.s4cc.symbolics.com>]

[Sorry this response has been so long in coming, but the mailer just
informed me today that it hadn't been delivered.]

    Date: Thu 14 May 87 12:01:04-CDT
    From: Art Flatau <CMP.FLATAU@R20.UTEXAS.EDU>

    Has anyone done any experimentation to see how much the Optimize World
    command reduces paging.  It seems to greatly increase the size of
    the world, so I'm interested in seeing how much this buys me.

    I saved an incremental world (in Genera 7.1) with and without doing an
    Optimize World.  The world was 9561 blocks with the Optimize World and
    4183 blocks without (obviously these are incremental worlds).

    Thanks for any info,
    Art

Some people at MIT had been complaining about this too, so I took the
opportunity to check it out (again).  In brief, it appears there may be
one bug which is causing the world to grow more than it should.  If it
turns out that this can be fixed, I will see it gets into the next
release.

The remainder of this message is a tutorial on Optimize World which
isn't in the documentation but should be.

Optimize World calls (SI:REORDER-MEMORY :INCREMENTAL T).  Because this
is incremental, only objects created since the last REORDER-MEMORY are
reordered.  Distribution worlds have SI:REORDER-MEMORY performed on them
as one of the last steps in the distribution process, so if you load a
system into a distribution world and then optimize it, only the newly
created objects will be moved around.  Since these objects are new, they
would be in the IDS file anyway, so the IDS file size would not
increase.  Similarly, any pages which reference the new objects would be
in the IDS file anyway, so transporting the objects to new addresses
should not have a large effect.

You'll notice by the caveats in the previous paragraph that there are
some conditions on what to do to keep IDS file size down.  Specifically,
if you load a system to a distribution, save an IDS, then boot it, load
another system, optimize world, and save a second-generation IDS, the
final IDS will be much larger than it would have been without the
optimize world.  This is because the second-generation IDS file will
have to supersede practically all the additions to the first.

Another caveat is that modifications to programs in the parent world
cause additional pages to be modified, increasing IDS size regardless of
whether Optimize World is used.

Finally, you should expect that IDS size will always increase if you
Optimize World, compared to what it would have been without Optimize
World.  Although ideally the effect is small, it is nevertheless
positive.  If IDS file size is your overriding concern, then you should
not use Optimize World.

As for the effects of Optimize World, that is directly dependent on how
much your application pages.  It affects only the working set and paging
time of applications, not the run time.  If your application hardly
pages, you probably don't need to optimize.  Similarly, since the core
world is already optimized, if the paging in your application is due
primarily to system functions, then optimizing the world would be
pointless.

On the other hand, for systems which page heavily, and for which the
working set is concentrated in their static objects (such as compiled
functions, flavors, symbols, property lists, etc.), then Optimize World
can have a significant advantage.  During development many programs were
observed to speedup by more than 3x on typically configured machines.
Some benchmarks where significant speedup were recorded were
compilations of large systems, scheduler/network working-set overhead,
and the time to boot the system.

If your application's paging is primarily due to dynamic structures,
then obviously a static object reordering can't help you.  I would
suggest a careful review of the dynamic data structures used if this is
the case.

In a future release, Optimize World will give you finer control over
these trade-offs.

In summary, there is no easy answer.  If Optimize World speeds up your
application significantly, and the increase in IDS size is acceptable,
then you should use it.  If not, you shouldn't.  Even if you don't,
however, you still have the benefit of the Optimize World which was done
on the system before it was released from Symbolics.