[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Gabriel benchmarks, HP DN10000 and others
>From att!MCC.COM!ai.gooch Wed Mar 7 15:34 CST 1990
>Received: by att.att.com; Wed Mar 7 16:12:19 1990
>Received: from KANGCHENJUNGA.ACA.MCC.COM by MCC.COM with TCP/SMTP; Wed 7 Mar 90 15:34:31-CST
>Date: Wed, 7 Mar 90 15:34 CST
>From: William D. Gooch <ai.gooch@MCC.COM>
>Subject: Re: Gabriel benchmarks, HP DN10000 and others
>In-Reply-To: The message of 6 Mar 90 14:41 CST from email@example.com
> Date: Tue, 6 Mar 90 15:41 EST
> From: firstname.lastname@example.org
>> The Symbolics numbers you gave are all quite slow compared to what I've
>> seen before for the same machines. What was your testing procedure?
> Freshly booted optimized full-gc'ed worlds
> (not necessarily distribution worlds)
> in which there were no background processes running
> and no intermachine communications. Loaded
> sys:examples;gabriel-benchmarks.bin from the file server
> (local machine in the case of the 3675 and the DN10000).
> Ran the entire suite using (run-series-to-file).
> I had compiled the gabriel-benchmarks for the Symbolics and TI with
> declarations off, and for the DN10000 both on and off. In all cases,
> EGC was on and DGC was off.
>May I suggest (as someone who has done enough benchmarking to know
>not to take any benchmark results too seriously) that you modify your
>procedure as follows:
I appreciate the suggestions, but I don't think they affect the results.
>1. Run each trial more than once, until you get consistent results.
>This eliminates inconsistencies due to up-front paging in of the code,
>etc. and creates a more realistic test.
This makes sense. However, when I ran the set twice on the 3650 I got
(nearly) the same results, and concluded that code paging wasn't an
issue. This is probably because the SYS:EXAMPLES;GABRIEL-BENCHMARKS
code automatically reruns the tests several times.
>2. On the Symbolics, use without-interrupts around each individual
>test. Even though it may appear that no background processes are
>active, there is usually a noticeable background overhead.
SYS:EXAMPLES;GABRIEL-BENCHMARKS performs this automatically too, if
I understand the code correctly.
>If you do these things, you should see both an overall improvement of
>your times and a better match of the 3650 results with the others.
>There should be no need to freshly boot or to use optimized worlds for
>the Gabriel benchmarks if you follow these suggestions.
>BTW, since the Apollo doesn't have and Ephemeral GC, shouldn't the GC
>just be left off in all cases to make a fair comparison (especially if
>you aren't using without-interrupts)? The Boyer trial is the only one
>which should be affected by GC in any case.
Since the version of Domain/Common Lisp
that was tested is a beta release, I may not be at liberty to discuss
what it does or does not do. I'll check with the (ack, pooey 8^)) lawyers.
AT&T Bell Laboratories
Whippany, NJ 07981