document examiner versus printed manuals -- a race or what?

[lakin@csli.Stanford.EDU (Fred Lakin)]

Yes, I agree that "the design of a test that is supposed to compare
paper with an on line system is far from trivial," and the
non-triviality would certainly include but not be limited to careful
question selection.

There is of course a tension between doing a Test in a neat, scientfic
way, and doing Something (a race) in a scruffy, casual way. 

The problem is that on one hand, there isn't time or energy to put
together a Test; and on the other hand the Something (race) would have
a lot of uncontrolled factors and then the outcomes might be abused
later, using them as results or data and making claims.

I guess I would vote for the Race.  Everyone would just have to
promise not to use the outcomes in bad ways ... except perhaps as a
rough sketch for designing a Test later. Interested parties (esp
systems administrators) might submit lists of questions they get a
lot; they also might include their bias if any toward either medium.

f