Re: Gabriel benchmarks, HP DN10000 and others

[ai.gooch@MCC.COM (William D. Gooch)]

    Date: Tue, 6 Mar 90 15:41 EST
    From: davel@whutt.att.com

	    The Symbolics numbers you gave are all quite slow compared to what I've
	    seen before for the same machines.  What was your testing procedure?
	
    Freshly booted optimized full-gc'ed worlds 
    (not necessarily distribution worlds)
    in which there were no background processes running
    and no intermachine communications. Loaded 
    sys:examples;gabriel-benchmarks.bin from the file server
    (local machine in the case of the 3675 and the DN10000).
    Ran the entire suite using (run-series-to-file). 
    I had compiled the gabriel-benchmarks for the Symbolics and TI with 
    declarations off, and for the DN10000 both on and off.  In all cases,
    EGC was on and DGC was off.

May I suggest (as someone who has done enough benchmarking to know 
not to take any benchmark results too seriously) that you modify your
procedure as follows:

1.  Run each trial more than once, until you get consistent results.
This eliminates inconsistencies due to up-front paging in of the code,
etc. and creates a more realistic test.

2.  On the Symbolics, use without-interrupts around each individual
test.  Even though it may appear that no background processes are
active, there is usually a noticeable background overhead.

If you do these things, you should see both an overall improvement of
your times and a better match of the 3650 results with the others.
There should be no need to freshly boot or to use optimized worlds for
the Gabriel benchmarks if you follow these suggestions.

BTW, since the Apollo doesn't have and Ephemeral GC, shouldn't the GC
just be left off in all cases to make a fair comparison (especially if
you aren't using without-interrupts)?  The Boyer trial is the only one
which should be affected by GC in any case.