random i/o tools

[p2@Porter.ASL.dialnet.symbolics.com (Peter Paine)]

    [...]
    Does anyone in the SLUG community have or know of Symbolics 
    software that can be used for doing random file i/o?  I am doing
    some database work and need tools for disk based hashing, B-trees, etc. 
    to integrate in the software I am developing.  I am aware that similar
    software components are available in Statiuc, but I need random i/o
    utilities that are not embeeded in a proprietary product.

Here is some code from an old system of ours, it uses DIRECT streams as a very fast 
bidirectional stream into disk store. You will notice in the documentation that you
are meant to open a file of :element-type 'string-char, well this works well on the
local disk, but I could not make it access a remote disk: hence the use of 
(unsigned-byte 8) etc.

If your data is fixed format (ours was all integer, each number occupying a 6 
character field), you can use a "bstream" to step and jump around randomly in the 
disk reading and/or writing as you go.
									  
I know the code is a grubby and uncommon, but it may give you ideas.

;;; -*- Mode: LISP; Syntax: Common-lisp; Package: USER; Base: 10 -*-
;;; ------------------------------------------------------------------
;;; B1functions0 handle bidirectional communication with fixed format stack file
;;; The file stream: 2#<FS:NFILE-BIDIRECTIONAL-DIRECT-BINARY-STREAM "P:>p2>data>test>filename1.text.newest" 17042334>
0;;; BIDIRECTIONAL-DIRECT-BINARY-STREAMs should be of :element-type 'string-char,
;;; however this fails over chaos link to non local hosts LMFS. So we use (unsigned-byte 8).
(defmacro 3WITH-STACK-INFO-BSTREAM0 ((stream-variable card-pathname) &body body)
  "Bind stream-variable to bidirectional-direct-binary-stream of card-pathname stack
   Note: bidirectional-direct-binary-stream is of :element-type (unsigned-byte 8)"
  `(with-open-file (stream ,card-pathname
			   :element-type '(unsigned-byte 8)
			   :if-exists :overwrite
			   :if-does-not-exist :create
			   :direction :io
			   :direct T)
     (let ((,stream-variable stream))
     (send stream
	   :read-bytes
;	   (getf (rest (send stream :properties)) ;; is this a safe way to get file length?
;		 :length-in-bytes)
	   (send stream :length))
     . ,body)))

(defun 3BREAD-LINE0 (bstream)
  "Read a line of bstream (8 bit byte), returning it as a string"
  (loop with string = (make-array 0 :element-type 'string-char :adjustable t :fill-pointer T)
	for byte = (read-byte bstream ())
	for char = (and byte (code-char byte))
	until (or (null char) (char-equal char #\return))
	do (vector-push-extend char string)
	finally (return (unless (zerop (length string)) string))))

(defun 3BWRITE-STRING0 (bstream string)
  "Write string to bstream"
  (loop	for char being the array-elements of string
	do (write-byte (char-code char) bstream)))

(defun 3BWRITE-STACK-HEADER0 (card-list card-pathname)
  (with-stack-info-bstream (bstream card-pathname)
    (bwrite-string bstream (format nil "~(~a~)" card-list))))

;;; Maybe reimplement as :append stream opening option
(defun 3ADD-CARD-TO-STACK0 (bstream data)
  "Hopefully we are at the end of file"
  (send bstream :set-pointer
	(getf (rest (send bstream :properties)) :length-in-bytes))
  (bwrite-string bstream (format nil "~S~%" data)))
;;; ------------------------------------------------------------------
<= p2