Re: MIPS rate (a resend)

[Reti@STONY-BROOK.SCRC.Symbolics.COM (Kalman Reti)]

    Date: Tue, 24 Sep 1991 11:11 EDT
    From: moon@cambridge.apple.com (David A. Moon)

    Raw MIPS numbers don't mean much when comparing machines with very different 
    architectures, you need to compare performance on applications that interest 
    you.  
Dave beat me to the punch, I was going to say the same thing.  In particular, the
MIPS model assumes that all "instructions" (or at least all frequently executed
instructions) take the same amount of time.  This is not true of many of the existing
standard architectures, and even less true of Lisp machines where, on some models at least,
a single instruction implements membership in an arbitrarily long list.
					     
Comparing MIPS on one machine to MIPS on another machine further assumes that the
amount of actual processing work done by the instructions on each machine type is
comparable.  This is also not true w.r.t. the Lisp machine where, for example,
the add instruction is type-checking both operands as well as adding. (This is
a multiple-instruction sequence on more standard architectures.)

	  Use Gabriel benchmark figures, and multiply by whatever Gabriel-to-MIPS 
    conversion factor you feel produces the results you want.  For instance if an
    authoritative source has told you that machine X is a 7.5 MIPS machine, and
    the geometric mean of the time for the half dozen principal Gabriels on 
    machine X is 4.1, then use a conversion factor of (/ 7.5 (/ 1 4.1)) = 30.75.  
    If machine Y does the Gabriels in 3.5 seconds, it's a (/ 30.75 3.5) = 8.8 MIPS 
    machine.

    I am sure there are other methods of calculation which will produce wildly 
    different results.  Don't use the 30.75 conversion factor I quoted above, I
    made it up out of air and smoke.  Calibrate it yourself.

Dave is suggesting you use the Gabriel benchmark programs because these are
"applications" that run on a wide variety of platforms more or less identically.
However, whether the result of this exercise says anything about how fast your
intended applications will run depends upon how close a model of your application
the Gabiels are.  Many customers have found that they aren't too close, mostly
because they are small and deal with a limited amount of data.  The performance
of real-world applications tends to depend a lot more on factors like paging and
I/O performance, the same things that make them difficult to use as a benchmark
suite on lots of platforms.