Re: PCL benchmark

[Gregor.pa@Xerox.COM]

    Date: Tue, 18 Oct 88 18:46:56 PDT
    From: Chris Burdorf <burdorf@rand.org>

    I got rid of the formats and compiled with safety set to 0 and
    speed at 3.  I got the following times:

Gathering these times into one table we get:

                                20 cars        50 cars

   ERNIE                        1.000          2.686

   PCL in AKCL                  8.483 (x8)    26.300 (x10)
   (speed 3, safety 0)
 
   PCL in Allegro               2.020 (x2)     6.340 (x2.4)
   (default compiler settings)

In these numbers, the PCL version of your program takes from 202% to
979% the time of the Ernie version.

Judging from these numbers, I would say that you should start writing
the rest of your code in CLOS.  If the program you sent is an accurate
measure of what your programs really do, CLOS is going to do better for
you than Ernie.

Why do I say this:

- You claim that Ernie is finely tuned for performance.  I
  believe you.  I claim that PCL is not operating anywhere
  near its "theoretical" performance.  Believe me.

- The best compilation you were able to give PCL in Allegro
  was with default compiler settings.  I am not sure, but I
  think this has a significant negative effect.  I will look
  into this.  I would also be interested in knowing why you
  couldn't compiler it with speed 3 and safety 0, this is a
  bug which should be fixed.

- From looking at your program, I believe it depends more
  than anything else on the performance of reader generic
  functions (those generic functions which have automatically
  defined reader methods on them).  Those will go considerably
  faster in more tuned versions of PCL.  In your Franz version,
  they will probably be 2 - 3 times faster.  In your KCL version
  they may be as much as 7 - 8 times faster.

- The next most critical number for your program is generic
  function call overhead.  In more tuned ports of PCL this
  will improve the same way that reader generic functions
  will (2 - 3 times in Allegro, 7 - 8 times in KCL).


    I can't tell you much about ERNIE, because it is funded by DOD
    and I don't want to get tried for treason.  It's basically a
    reimplemtation of FLAVORS.  It has practically everything Flavors
    has, but it was designed especially to be fast.

This does and doesn't make sense to me.

   Does:    The Ernie performance numbers don't look all that
            good, they seem to be what one would expect from
            flavors.

   Doesn't: If I were designing something with efficiency as my
            single I wouldn't start from Flavors.  If efficiency
            were not my my single goal, I would start with CLOS.
            CLOS has the go good ideas from Flavors (and other
            languages as well) and has been carefully designed
            to allow efficient implementation.

Another point is that CLOS, with its metaobject protocol, can allow more
efficient user controlled, "staticizing" or "block compilation".

			      
			 My main reason for sending out this comparison
    of ERNIE and PCL was not to try and convert people to ERNIE, but to
    try to  stir a dialog on efficiency.  Hopefully PCL will be fast for
    all of us: rich or poor. 

Yes, this is an important dialog.  As for your cost concern, I believe
PCL will run well in KCL as well as in more expensive lisps.
-------