poor make-pipe-stream performance

[marcus@ee.pdx.edu (Marcus Daniels)]

I'm archiving big records (up to 1.6 meg) as structures, using the
#S() syntax.  It is generated by a simple database interface program
that communicates through standard I/O.  The records consists 
primarily of double-vectors (6 sets, of 300-30000 elements).

My plan was to use make-pipe-input-stream to `read' from this program.

I've found that pipe I/O performance is not as good as AKCL.  Reading
directly from a file yields equivalent performance to AKCL, so I 
don't believe it is a O.S. thing (and there is plenty of resident
memory free in all cases).

CLISP and AKCL are both compiled with gcc-2.5.6 -O6 on Linux.  CLISP
doesn't use shared memory due to the 4MB limitation (in the following
example the memory use is around 7.5 MB).

Here were the results for the largest record (27951 * 6 * double-vector + 
3 identifier fields).
				Real-time 	`Run-time'
CLISP Straight (local) file: 	37.6		35.1
CLISP MKFIFO file:		258.3		151.04
CLISP MAKE-PIPE-INPUT-STREAM	232.9		145.27
AKCL Straight (local) file:	33.0		N/A
AKCL RUN-PROCESS		58.0		N/A

The CLISP MKFIFO/MAKE-PIPE exec'ed way the shell... So the fact that AKCL
uses execvp with no shell wouldn't seem to matter.

Also, I played with using dd to reblock stuff for CLISP, but that didn't
seem to accomplish anything..

I'd be willing to pay the 25 second penalty for such records, if 
I could use pipe, but not a 3 minute penalty.  Are there any buffering
parameters I might play with?  What else could account for this discrepancy?

The subshell approach was intended to be tolerable, hopefully temporary,
substitute for a socket based or Postgres backend (perhaps something like
Picasso or CLING).  Has anyone ever worked out a simple socket oriented 
(i.e. database) interface for CLISP?

I noticed that the only socket support CLISP includes is apparently for
the sake of CLX -- not for general use.  Is there any (technical) reason
not to provide such an interface (other than it could be difficult or
impossible to provide GPL'ed on all platforms).  But a DJGPP network
capable CLISP would come in _very_ handy for all the DOS bound PC crud
around here!

regards,
Marcus Daniels