[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Date: Thu, 22 Feb 90 19:04 PST
From: Kalman Reti <Reti@STONY-BROOK.SCRC.Symbolics.COM>
Date: Thu, 22 Feb 90 14:29 EST
From: firstname.lastname@example.org (Barry Margolin)
If you can get exclusive use of a system, this is a reasonable way to do
benchmarks. However, the TIME macro doesn't know that you've arranged
this, so it reports the two numbers.
Traditionally, "CPU time" refers to the amount of time the processor
spends executing within a given process; in most systems this is
implemented by the scheduler noting the time when it schedules a process
to run, noting the time when it next deschedules that process, and
adding the difference to a total for the process. On timesharing
systems this is useful since it factors out the load of the system
somewhat; a program should take approximately the same amount of CPU
time no matter how many other users are using the system. Systems
differ in whether they include time spent in the kernel in this amount;
some systems distinguish "system time" and "user time". Sometimes it's
useful to disregard system time, because it is more affected by
activities of other users (for example, if you're timing file access,
the system time will depend on whether another user had recently
accessed the same or a nearby file).
An excellent summary; I'd only like to add that even this this definition
of CPU time can produce widely varying results because of paging. For that
reason the KL-10 computed CPU time by counting memory references (it had
hardware for that) which resulted in the same 'CPU time' for every execution
of the same program whether it never encountered a page fault or if it took
one on every single memory reference (impossible, but you get the idea).
I think that you missed my point. I know what CPU time is and I know why it is
reported and why people use it. I just reiterate my point that personally I
find elapsed wall clock time to be the most appropriate measure of system
performance. I know that it can vary depending on many factors such as system
load, network load, and paging quirks. BUT ALL OF THESE AFFECT HOW LONG I MUST
WAIT TO GET MY RESULT. And that is all that counts to me. Besides, few people
use timesharing systems anymore today. Most people use single user machines
(even if they happen to be Unix boxes which can support timesharing, they are
apportioned one per person and typically run one user task at a time) so most
of the arguments of using CPU time to get load invariant benchmark data are
overweighed by the response time argument.
On single-user workstations such as the Lisp Machine it is less
necessary to factor out the affects of other processes, because the user
is believed to have more control over them than on a timesharing system.
In Genera you can use WITHOUT-INTERRUPTS or PROCESS:WITHOUT-PREEMPTS to
hog the processor when you want to factor out other processes, but
ordinary users on timesharing systems generally can't get exclusive use.
CPU time is supposed to be close to the time you'd see if yours were the
only process on the system. If you just want to know whether 3650 or
Sparcstation-1 processors are faster at addition this is usually the
right way to measure it.
The time spent servicing network interrupts can be significant, especially
on heavily-used networks, so in addition to WITHOUT-INTERRUPTS-style control,
I unplug my network.
There are, of course, times when CPU time is completely worthless. For
instance, if you're timing operations that use network facilities or
back-end processors, you don't want to factor out the time spent waiting
for the server. Or if you want to know how well a program performs
under various system loads you want to measure its real time.
- From: barmar@Think.COM (Barry Margolin)
- From: Reti@stony-brook.scrc.symbolics.com (Kalman Reti)