Re: MSPS-game (Re: PCL benchmark)

[hdavis.pa@Xerox.COM]

I don't think Ida's code is quite fair to pcl, or lives up to its promises.
Despite his claim that the pcl half is "25% classical method with two
argument, 50% with default arguments, 25% multi-method", actually, the
multi-method discriminator code has to be called at every generic function
call.  Thus, this may be a fair test of multi-methods, but pcl optimizes
for the common case of single-argument method discrimination.  In addition,
the code does a make-instance with every generic function call but not for
the regular function calls, mixing in this expensive operation with the
generic function call.  We should sort these cases out.  Finally, it is not
clear how inheritance is being tested in the code.

Just to test multi-methods vs. discrimination on a single argument vs.
regular function call (although that's not really fair), I ran the
following simple code in Xerox CommonLisp:

(in-package "BERK" :use '("LISP" "PCL"))

(defclass foo () ())
(defvar *foo* (make-instance 'foo))

(defclass bar () ())
(defvar *bar* (make-instance 'bar))

(defmethod call-me ((x foo) y) 
	(if (zerop y) nil (call-me *bar* (1- y))))
(defmethod call-me ((x bar) y) 
	(if (zerop y) nil (call-me *foo* (1- y))))

(defmethod mm-call-me ((x1 foo) (x2 bar) y) 
	(if (zerop y) nil (mm-call-me *bar* *foo* (1- y))))
(defmethod mm-call-me ((x1 bar) (x2 foo) y)
	(if (zerop y) nil (mm-call-me *foo* *bar* (1- y))))

(defun call-me-foo (x y) 
	(if (zerop y) nil (call-me-bar *bar* (1- y))))
(defun call-me-bar (x y) 
	(if (zerop y) nil (call-me-foo *foo* (1- y))))

Results were:

(time (call-me *bar* 500)) ==> 		42 ms
(time (mm-call-me *bar* *foo* 500) ==>	first: 129 ms
					others: 61 ms
(time (call-me-foo *foo* 500)) ==>	20 ms

and, for good measure,

(time (make-instance 'foo) :repeat 500) ==>	1602 ms
(time (pcl::*make-instance 'foo) :repeat 500)=>	3960 ms

make-instance creates 2 lists, 1 oned-array, and 1 instance per call.
*make-instances creates 12 lists, 1 oned-array, and 1 instance per call.

So I think it's fair to say that Ida's numbers largely reflect the time for
make-instance, and not really the overhead for generic function calls.  I
must say I was pleasantly surprised to find that generic function calls
only take 2-3 times a regular function call in XCL.  Is this true in other
Lisps as well?  How about tests for slot accessors?

	-- Harley

PS My *pcl-system-date* is "9/21/88 (beta) Pre-Workshop PCL"