Re: Gabriel benchmarks, HP DN10000 and others

[davel@whutt.att.com]

>From att!MCC.COM!ai.gooch Wed Mar  7 15:34 CST 1990
>Received: by att.att.com; Wed Mar  7 16:12:19 1990
>Received: from KANGCHENJUNGA.ACA.MCC.COM by MCC.COM with TCP/SMTP; Wed 7 Mar 90 15:34:31-CST
>Date: Wed, 7 Mar 90 15:34 CST
>From: William D. Gooch <ai.gooch@MCC.COM>
>Subject: Re: Gabriel benchmarks, HP DN10000 and others
>To: davel@whutt.att.com
>cc: slug@Warbucks.AI.SRI.COM
>In-Reply-To: The message of 6 Mar 90 14:41 CST from davel@whutt.att.com
>Message-ID: <19900307213434.4.GOOCH@KANGCHENJUNGA.ACA.MCC.COM>
>
>    Date: Tue, 6 Mar 90 15:41 EST
>    From: davel@whutt.att.com
>
>>    The Symbolics numbers you gave are all quite slow compared to what I've
>>    seen before for the same machines.  What was your testing procedure?
>>
>    Freshly booted optimized full-gc'ed worlds 
>    (not necessarily distribution worlds)
>    in which there were no background processes running
>    and no intermachine communications. Loaded 
>    sys:examples;gabriel-benchmarks.bin from the file server
>    (local machine in the case of the 3675 and the DN10000).
>    Ran the entire suite using (run-series-to-file). 
>    I had compiled the gabriel-benchmarks for the Symbolics and TI with 
>    declarations off, and for the DN10000 both on and off.  In all cases,
>    EGC was on and DGC was off.
>
>May I suggest (as someone who has done enough benchmarking to know 
>not to take any benchmark results too seriously) that you modify your
>procedure as follows:
I appreciate the suggestions, but I don't think they affect the results.
>
>1.  Run each trial more than once, until you get consistent results.
>This eliminates inconsistencies due to up-front paging in of the code,
>etc. and creates a more realistic test.
This makes sense.  However, when I ran the set twice on the 3650 I got 
(nearly) the same results, and concluded that code paging wasn't an 
issue.  This is probably because the SYS:EXAMPLES;GABRIEL-BENCHMARKS
code automatically reruns the tests several times.
>
>2.  On the Symbolics, use without-interrupts around each individual
>test.  Even though it may appear that no background processes are
>active, there is usually a noticeable background overhead.
SYS:EXAMPLES;GABRIEL-BENCHMARKS performs this automatically too, if
I understand the code correctly.
>
>If you do these things, you should see both an overall improvement of
>your times and a better match of the 3650 results with the others.
>There should be no need to freshly boot or to use optimized worlds for
>the Gabriel benchmarks if you follow these suggestions.
>
>BTW, since the Apollo doesn't have and Ephemeral GC, shouldn't the GC
>just be left off in all cases to make a fair comparison (especially if
>you aren't using without-interrupts)?  The Boyer trial is the only one
>which should be affected by GC in any case.

Since the version of Domain/Common Lisp
that was tested is a beta release, I may not be at liberty to discuss
what it does or does not do.  I'll check with the (ack, pooey 8^)) lawyers.

David Loewenstern
AT&T Bell Laboratories
14B-253 
Whippany, NJ 07981
email: davel@whutt.att.com
at&t: 201-386-6516