XL1200 "Benchmark"

[pao@ai.mit.edu (Patrick A. O'Donnell)]

We recently timed the XL1200 against two Sun workstations and a 3650,
and we thought the folks on SLUG might be interested in seeing the
results.  This was not a "careful" benchmark, and the program was not
particularly tuned for any of the machines.  (It was developed on
3600-series machines.  Many numerical declarations have been added for
better performance under Lucid.)  We feel this represents a good
estimate of the relative speeds we would see in our development use on
the various machines.

Handey, the program we timed, is a robot task planning system, with
intermingled symbolic and numeric computation.  Nearly all the numeric
computation is single-precision floating point.  (Some sections use
integer arithmetic.)

For all four machines, we timed compilation of the whole Handey system
(about 4.5 megabytes).  We then timed execution of two Handey
demos---each of which compute a path to grasp an object in the robot's
workspace taking other obstacles in the world into account.  The
difference between the "red-l" and the "plywood" demo is simply shapes
and number of the objects in the workspace.  Both demos were run both
with and without explanatory display.

Without further ado, the numbers:

Date: 9-12 May 1990
Handey version: nhandey of the above date.

Machines:

soggy-fibers	Sun 4/370, 56 Mb main memory, local disk.
Sparcstation	a standalone Sparcstation 1, 16 Mb main memory, local disk.
sluggo		3650, 3.5 Mw main memory, network file access.
XL1200		XL1200, 8 Mw main memory, local file access.

Soggy-fibers, Sluggo, and the XL1200 were all on the network; the
Sparcstation was standalone.  Sluggo, the Sparcstation and the XL1200
were otherwise idle; soggy-fibers had other user processes extant, but
they were idle.

Compilation info:

For the SUN workstations, the Lisp was Lucid, using the production mode
compiler, compiling with the default optimization settings (speed = 2,
safety = 3).  Setting speed to 3 or safety to 0 had no appreciable
effect on the timings.

Tasks:

1) Compile Handey system.  (for SUN, the times are for the production mode compiler)
2) red-l demo (no display)
3) red-l demo (full display)
4) plywood demo (no display)
5) plywood demo (full display)

All times are total wall time, in seconds, to the nearest 5 seconds.
The numbers in parentheses are the number of page faults rounded to the
nearest hundred or so.

		Task
					red-l				plywood
Machine		compile		no disp		disp		no disp		disp
-------------------------------------------------------------------------------------------
XL1200		860 (3800)	 55 (11)	 85 (??)	 65 (0)		 90 (0)
soggy-fibers	840 (2900)	 80 (40)	160 (600)	 95 (70)	165 (1000)
Sparcstation   1550 (44000)	110 (1300)	160 (1200)	105 (1000)	150 (800)
sluggo	       3600 (1100000)	250 (24000)	345 (43000)	265 (36000)	345 (45000)


This table is the same as the above, but normalized to the XL1200.

		Task
					red-l				plywood
Machine		compile		no disp		disp		no disp		disp
-------------------------------------------------------------------------------------------
XL1200		1.0		1.0		1.0		1.0		1.0
soggy-fibers	1.0		1.5		1.9		1.5		1.8
Sparcstation	1.8		2.0		1.9		1.6		1.7
sluggo	        4.2		4.5		4.1		4.1		3.8