CL Benchmarks Using AutoClass II: Revisited

[taylor@CHARON.arc.nasa.gov (will taylor)]

Our benchmark is based upon AutoClass II, an unsupervised Bayesian 
classification system for independent data, which is currently being used
by several researchers on real world problems.  It utilizes floating point
operations and vector manipulation to a high degree.  The code was refined
to minimize garbage collection: the basic search loop is static with respect
to space allocation.  Thus excessive garbage can only be generated by an
operating system's implementation of math operations.  

The system was developed on the Symbolics and hence has no type declarations
(NO-DCL).  In comparing performance with Franz CL and Lucid CL, it should be 
noted that adding type declarations (DCL) increases performance by 10-30% on 
the Suns.

Franz and Lucid benchmarks on Sun platforms were compiled using optimizations
of speed 3, safety 0, and compilation-speed 0.  Compilation-speed was inadvertently
left at 3 during previous compilations, hence the Lucid and Franz cpu numbers 
are lower than those reported before. 

Comparing benchmarks between Symbolics operating systems and UN*X operating
systems is not straight-forward, since the Common Lisp function TIME returns
elapsed time on the Symbolics, and cpu time for Franz and Lucid on UN*X
systems (the garbage collection portion is reported for Franz CL).  So for 
the Franz CL timings, the times reported as cpu times are the total cpu time 
minus the garbage collection time.  The Symbolics cpu benchmarks were run 
under Genera 8.0 with PROCESS:WITHOUT-PREEMPTION.  The Symbolics elapsed 
benchmarks include garbage collection AND other non-essential system processes
such as network processing, so they are not truly reflective of the benchmark.
%-cpu and %-elap are referenced to the XL1200.


   cpu  %-cpu  elapsed  %-elap  configuration
-----------------------------------------------------------------------------
  5.30  100.0     5.60   100.0  Symbolics XL1200 (16mb RAM; FPA; Genera 8.0)

 13.86   38.2                   Symbolics 3675 (14mb RAM; FPA; Genera 8.0)

 14.35   36.9                   Symbolics XL400 (16mb RAM; FPA; Genera 8.0)

 14.53   36.5    17.26    32.4  Symbolics 3653 (16mb RAM; FPA; Genera 8.0)

 14.94   35.5	 21.92 	  25.5	Lucid CL 4.0.beta -Sun Sparcstation 1
				   (DCL; 16mb RAM; SunOS 4.0)
 18.29   29.0    20.56    27.2  Franz Allegro CL 3.1.13 - Sun Sparc 1 plus
			           (DCL; 16mb RAM; SunOS 4.1)
                 22.78    24.6  Symbolics UX400S (16mb RAM; Genera 7.4, Sun 
                                   SparcStation 370 (Sun 4.0)
	         23.04    24.3  Texas Instruments Explorer II (32mb RAM,
			           Release 3.2)
 21.25   24.9    24.39    23.0  Franz Allegro CL 3.1.13 - Sun Sparc 1	
			           (DCL; 16mb RAM; SunOS 4.1)
 21.8    24.3    25.6     21.9  Franz Allegro CL 3.1.13 - Sun Sparc 1 plus
			           (NO-DCL; 16mb RAM; SunOS 4.1)
 22.18   23.9    28.87    19.4  Lucid CL 4.0.beta -Sun Sparcstation 1
				   (NO-DCL; 16mb RAM; SunOS 4.0)
 26.1    20.3    30.7     18.2  Franz Allegro CL 3.1.13 - Sun Sparc 1
			           (NO-DCL; 16mb RAM; SunOS 4.1)
                 36.15    15.5  Symbolics MacIvory (20mb RAM; FPA;
	                           Genera 7.3I)

Thanks to Eric Benson of Lucid, Inc for help with the appropriate declarations and
running the Lucid cases.

=============================================================
Will Taylor       Sterling Software/NASA Ames Research Center
MS 244-17, Moffett Field, CA 94035  taylor@pluto.arc.nasa.gov
=============================================================