Allegro 4.1 on a Sparc 10 and other benchmark fables

[IRA@merry.radonc.washington.edu (Ira Kalet)]

I read with interest the comments about real life applications running not
as fast as expected on a Sparc 10 vs a Sparc 2.  We have a complex application
that involves interactive 3-d graphics, medical image display, a small
widget set and user interface kit that I wrote instead of using CLIM (it
uses CLX).  As we were about to purchase 6 new workstations, and we had
gone to extremes to make our code portable (easier in CL than most other
languages), we benchmarked our code on a variety of systems, including the
following:

DEC5000/240 Ultrix
IBM RS6000/340 AIX
Silicon Graphics Indigo XS24
Hewlett-Packard 720 HP-UX 8.07
Hewlett-Packard 735 HP-UX 9.0 (with a little help from Duane)
Sun Sparcstation 10/30 with GT graphics, SunOS 4.1
(the HP systems had CRX 8 bit graphics)

The Sparc 10, the SGI Indigo and the HP 720 were all about the same level
of performance, the DEC5000 was comparable on most operations involving
moving large amounts of data (pixmaps, arrays, etc.), and the DEC and IBM
were a lot slower in the most compute intensive parts.  The HP 735 was about
twice as fast as the nearest competitor.  The timings were all wall clock
timings since the application is pretty interactive, and we didn't have time
to sete up something more elaborate.  Particular points included:
The Sun was better at reading in large (25MB) data sets than any of the 
others, the IBM was the worst in pretty all aspects.

We also ran a floating point intensive Pascal program on all the above
systems, and got big surprises: the DEC5000/240 outperformed the Sparc 10
and the IBM by significant margins.  This flatly disagrees with the 
Spec FP and LINPACK numbers.  I believe our results reflect compiler
technology since our program is big and good optimization makes a difference.
On this HP came out on top and SGI second.  We saw a factor of 2 to 4 
difference from what would be predicted by the "standard" benchmarks.

We ordered 3 HP 735's and 3 HP 715's.

Benchmarking is indeed a black art but if you run your own code you can 
trust that the numbers are real.

Aside from HP-UX 9.0 we had no problem installing Allegro CL on all the 
machines, we made no modifications to any of our code, and all the vendors
we worked with were helpful, friendly and supportive (most of all Franz,
of course).

Oh, yes, the comparision with Sparc 2.  We ran the Pascal program on our
Sun 4/470 and it ran about 3 times faster on the Sparc 10 as on the 4/470.
I just ran our wall clock lisp benchmarks also, and got varying results.
On file i/o the Sparc 10 is more than twice as fast.  On the interactive
graphics, I get a factor of more than 2 on some operations, a little less
than 2 on others, and on one - computing a new image with a new linear
gray map, the 4/470 and Sparc 10 take the same time, in fact the 4/470 is
a little faster.  Go figure.

Ira Kalet
Radiation Oncology Dept. RC-08
University of Washington
Seattle, WA 98195