Request for help with paging

[whuts!davel%att.att.com@RELAY.CS.NET]

From: PORSCHE.SCRC.Symbolics.COM!khs
>    Date: Tue, 14 Mar 89 20:08:41 EST
>    From: whuts!davel@att.att.com

>    We have written a system which appears to have a grave difficulty with
>    thrashing in the paging system.  When freshly loaded, the system
>    appears to spend perhaps 10% of its process time in the paging system,
>    but after about four hours of continuous running, paging system time
>    increases dramatically to 80% or so.  These results have been confirmed
>    with both the TIME macro and the Metering system.  According to the
>    metering system, there is no small subset of functions responsible for
>    most of the paging.

>It's usually the data structures that matter in paging, not the functions.

I guess I wasn't clear.  We didn't think that paging in the functions was
a problem (according to SI:SUMMARIZE-PAGE-TRACE, about 5% of the
page faults are to compiled-function-area).  We thought that no small
subset of functions were *responsible* for the paging -- that is, they
didn't reference many objects which caused page faulting.

For example, if we stored all of our objects in a single a-list, then
we would expect a huge amount of paging to be performed in the table
lookup function, but little to be spent in I/O, analysis of the objects,
etc.  In fact, what we see is paging being spread across many different
functions, not just the lookup functions.

>			 So far, we have found that our efforts to increase
>    locality of reference by using many areas have yielded minimal improve-
>    ment.  We had expected the problem to be solved by the use of areas;
>    we have a physical memory of 6MW, each area is smaller than 
>    4MW or so for the first five hours of the run, and references between 
>    areas should be uncommon.

>The point of using areas is not to minimize interarea references; if
>that were the case you'd do best with one area.  The point is to
>optimize main memory usage by concentrating useful objects.  That is,
>objects that are likely to be used at the same time should be located
>together.  This minimizes the amount of main memory wasted storing
>objects that aren't useful at the moment but happen to be on the same
>page as useful objects.

I again apologize for making myself unclear.  The idea was to concentrate
useful objects.  Yet clearly, the number of interarea references is 
important; otherwise, you could improve paging performance by
placing each object in a separate area!

>Genera uses a fairly standard LRU approximation for page replacement;
>a clock algorithm supported by hardware maintained reference tags.
>3600 and Ivory machines make approximately the same page replacement
>decisions, but the Ivory implementation uses fewer CPU cycles to do so.

Thank you.  I've been trying to get this answer for a week.

>Anyone who's used a virtual memory system has had problems with
>paging at some point.  They're usually solvable with patient analysis
>followed up by careful design and implementation, but it's often
>cheaper just to buy more memory boards.  I believe there are several
>people in the Symbolics consulting group with experience at
>optimizing paging performance; you might consider their services.

We'd love to buy more memory boards;  however, the 3675 won't take
any more.

>Here's a function to describe main memory usage by area:
Thank you.  

    David Loewenstern
    <davel@whuts.att.com; backbone!{moss || ihnp4}!whuts!davel>
    AT&T Bell Laboratories
    14B-253
    Whippany, NJ 07981
    201-386-6516