Sunday December 9,1979 FM+5D.18H.52M.12S. LISP 1914/COMPLR 904 - JONL, RWK - WARNING! The two new macro-characters, items 1 and 2 below, may become standard someday (set up as macros in the initial lisp environment); anyone who will have trouble accommodating should send mail to BUG-LISP. 1) +INTERNAL-#-MACRO is autoloadable, for use as a reader-macro-character function for "#"; fully compatible with NIL/LISPM usage. 2) +INTERNAL-/"-MACRO is defined for use in making " a string macro character. TTYSCAN-STRINGERS| helps TTY-rubout processing in string-like sequences. 3) Two **INCOMPATIBILITIES**: Carriage-Return, no longer invisible, is now a "white-space" character. TERPRI is initially non-null, meaning no automatic insertion of <cr>'s. 4) PROG1 -- Like PROG2 but return first argument instead of second. 5) SETF is an FSUBR like the LISPM's general update operator DEFSETF is an autoloadable macro to help one extend the SETF range. 6) As on LISPM, PUSH and POP have been extended to use SETF instead of SETQ. 7) INCLUDEF is a subr version of INCLUDE, for the sake of transportable code. 8) A new "Software device" convention for NAMELIST-series functions. Namelists of the form ((LISP) * *) refer to the maclisp system area. 9) LDB and DPB and other new AUTOLOADable software as fasl file on LISP: files ((LISP) DEFVST FASL), ((LISP) MACAID FASL), ((LISP) LODBYT FASL) 10) VSAID - VECTOR support package; STRING - support and NILAID on LISP: 11) (STATUS FILESYSTEM-TYPE), (STATUS OPSYSTEM-TYPE), and (STATUS SITE) Warning! (LAST (STATUS FEATURES)) for non-ITS systems will be different. 12) Some things that failed to be documented previously about DEFUN/DEFUN& &WHOLE - DEFMACRO now permits a keyword &WHOLE, to get at whole call form DEFUN reverts to DEFUN& "if necessary"; e.g. for keywords and destructuring 13) FRETRY, a new function which is in many ways like FRETURN. 14) The setting of "+" in the toplevel and break loops modified. 15) SPRIN1, a Pretty-printing version of PRIN1 autoloadable from GRINDEF 16) CLEAR-INPUT and CLEAR-OUTPUT on SFA's now works 17) USERATOMS-HOOKS, for compiling user-defined "SQUIDified" datatypes 18) EOF-COMPILE-QUEUE, for queueing forms to be "done" at the end of file. ____________________________________________________________________________ ** NOTE ** In the commentary which follows, this marker marks sections ** NOTE ** explaining features not yet present on the LISP machine. ** NOTE ** At some point in the future it is expected they will be added, ** NOTE ** but in the interim, people writing code for both systems should ** NOTE ** be careful. 1) +INTERNAL-#-MACRO is autoloadable, for use as a reader-macro-character function for "#"; fully compatible with NIL/LISPM usage. +INTERNAL-#-MACRO is provided with an autoload property such that if the user does (SETSYNTAX '/# 'SPLICING '|+INTERNAL-#-MACRO|) he will get a MACLISP/LISPM/NIL compatible version of the read macro character for #. Lately, this "generalized" macro character has been especially useful for conditionalizing out lines of code based on which dialect of lisp the code is being read into. Needless to say, it would be nice if users could begin to accommodate this macro-character definition, so that it can turned initailly on some day - it currently is set on by LISPM and NIL. The read-time meaning of a "#" instance is determined by a "dispatch" character, which immediately follows the "#" (except when there is a numeric "argument", in which case the base-10. digits occur between the "#" and the dispatch character, or if there is a "control,meta" argument in which case the argument is between the "#" and the dispatch). The dispatch character is completely programmable (by the user too) in essentially the same way that macro characters in general are programmable. The function SETSYNTAX-SHARP-MACRO is like SETSYNTAX, but is for the meanings of the character following a #. However, the associated function must take one argument, which will be bound to any "argument" as indicated above - null for no argument, a fixnum if the argument is a digit string, and a symbol if the argument is one of ^B ("CONTROL"), ^C ("META"), or ^F ("CONTROL-META"). DEFSHARP, an autoloadable macro, provides an easy way to set up "macro character functions", to which the general # macro character function will dispatch, depending on the character immediately following the #. (DEFSHARP /% (() ) (MACROEXPAND (/#SUB-READ () READ-STREAM))) defines (and activates using SETSYNTAX-SHARP-MACRO) a function to be run when #% occurs. There are several more options, for which the interested party is referred to the commentary in the source code near the definition of DEFSHARP. Currently, the following characters are set up as "SHARP-MACRO"s: / - Input the following character, and give out its ascii value - thus #/A yields 65. (101 octal). A "cntrl,meta" argument will add in amount to make a 9-bit ascii code, with "cntrol" adding 2^7, and "meta" adding 2^8; thus #/A yields 449. (701 octal). ^ - Input the following character, and XOR 2^6 into its ascii value - this generally has an effect similar to holding the control key down while typing that character. Any argument is ignored. . - Read-time ealuation of the next form. Any argument is ignored. Thus, (A #.(1+ BPORG) B) might read in like (A 125755 B). , - Load time evaluation of the next form. "#," is the same is "#." except when compiling, in which case a "SQUID" is returned. Any argument is ignored. % - Read-time macro expansion; thus #%(FOO 3) will be read in like #.(MACROEXPAND '(FOO 3)) ' - #'<foo> reads in like (FUNCTION <foo>). Any argument is ignored. * - Numeric read in the "natural base" or your machine - thus for the PDP10 and LISPM, #*xxxx* will read the digits "xxxx" in octal; likely some other machines will use hexadecimal. Any argument is ignored. [This is intended for use by machine dependent code.] \ - Following this is an alphabetic name of a character, and its ascii value is returned; as with "#/", a "cntrl,meta" argument will potentially convert to a 9-bit code. The correspondence between names and values is as follows: NULL - 0, BS - 8, BACKSPACE - 8., TAB - 9, LF - 10., LINEFEED - 10., VT - 11., FF - 12., FORMFEED - 12., CR - 13., RETURN - 13., NL - 13., NEWLINE - 13., ALTMODE - 27., SP - 32., SPACE - 32., RUBOUT - 127., DELETE - 127., ALPHA - 2, BETA - 3, EPSILON - 6, HELP - 2110. BELL - 7., FORM - 12., BACK-NEXT - 31. + - Next form should be a feature name, or boolean combination of feature names, and the form after that will be read in and flushed unless the specified featues are on the (STATUS FEATURES) list. E.g. (A #+SAIL B C) Will be "(A B C)" on the SAIL system but "(A C)" elsewhere. (A #+(and (or VSAID STRINGS) (not EXPERIMENTAL)) B C) As above, but the "B" is present only for non-experimental systems which have either a VSAID or STRINGS feature. - - Just like "+" above, but with the sense reversed. Thus #-{...} is equivalent to #+(not {...}) M - Equivalent to #+MACLISP N - Equivalent to #+NIL, except that additional NIL syntax is permitted in the forms to be passed over. For example, #B"10110" and #T and #(...) would normally be illegal for maclisp, but within the scope of a #N they are readable (for the primary purpose of correct parsing, since they would normally be thrown away; but if the NILAID pacakge is present, supporting a sort of NIL-in-maclisp, then a cooperative answer is returned). O - Octal reading - a number should follow Q - Equivalent to #+LISPM R - Temporary radix change for reading a number; e.g. #3R2010 reads in base 3 and returns 57. The "base" must be a number between (not inclusive) 1 and 37., expressed as digits in base ten; thus #16R... would indicate a hexadecimal notation. Due to a shortcoming of maclisp, if you want to use digits greater than 9, as in #16R-1A9 (= -425.), you must preceed the number with an algebraic sign. X - Hexadecimal reading - a number should follow On the ITS systems, the source code (or a link to it) will be in the file LIBDOC;SHARPM NIL 2) +INTERNAL-/"-MACRO is defined for use in making " a string macro character. TTYSCAN-STRINGERS| helps TTY-rubout processing in string-like sequences. The function defined for +INTERNAL-/"-MACRO returns an uninterned symbol which has been set to itself; as with "#", it is up to the user to do (SETSYNTAX '/" 'MACRO '+INTERNAL-/"-MACRO) if he wants. TTYSCAN-STRINGERS| is a variable set up to an a-list of start and stop (ascii values of) characters so that the system-supplied tty prescan function can know about which (read macro) characters are string-like delimiters. Of course, you may supply your own macro function for doublequote if you don't like the style of +INTERNAL-/"-MACRO. The compiler version of this function - +INTERNAL-/"-MACRO - in addition to the abovementioned action, puts a "+INTERNAL-STRING-MARKER" property on the symbol as well, so that it may output `(PROGN (SETQ ,symbol ',symbol) (PUTPROP ,symbol T '+INTERNAL-STRING-MARKER)) This allows a macro to use a string compiled into it in producing code which gets output to yet another FASL file, and have that final one still have the same characteristics of the original. Code which produces symbols for strings can put this property on when in the compiler, to cause this processing to happen for his strings as well. This saves the user from having to write his own USERATOMS-HOOK (see below). 3) Two **INCOMPATIBILITIES**: Carriage-Return, no longer invisible, is now a "white-space" character. TERPRI is initially non-null, meaning no automatic insertion of <cr>'s. This means that <cr> will terminate an atom, just like space does, and also that it is not necessary to "slashify" <cr>'s occurring within a vertical-bar or double-quote string. Actually, many persons may prefer to reset TERPRI to (), so that symbols will not appear to "wrap-around" at the end of a line; the only danger is that atoms (symbols or bignums) which are longer than the line-length will have a carriage-return inserted "in the middle" of them, thereby destroying read-back capability. 4) PROG1 -- Like PROG2 but return first argument instead of second. (PROG1 e1 e2 ... en) is like (PROG2 () e1 e2 ... en) So if you had this defined as macro in your files, you can relax now. 5) SETF is an FSUBR like the LISPM's general update operator. DEFSETF is an autoloadable macro to help one extend the SETF range. SETF is particularly useful with structure accessing macros, such as DEFVST (or other packages) would define. It's syntax is like a generalized SETQ. For example, assume there is a macro defined by (DEFMACRO SHIP-LABEL (X) `(CXR 6 ,x)) then (SETF (CADDAR A) 'BAR (CDR B) (LIST 4) Y 'BUZZ (SHIP-LABEL MYSHIP) 'JOLLY-ROGER) is equivalent to (PROGN (RPLACA (CDDAR A) 'BAR) (RPLACD B (LIST 4)) (SETQ Y 'BUZZ) (RPLACX 6 MYSHIP 'JOLLY-ROGER)) Five standard cases are "built-in" in the interpreter, but are redefinable with the generalized SETF expander package which will be autoloaded when used. a) symbols, in which case SETF acts just like SETQ; b) car/cdr operations up to depth 4, inversion is either a RPLACA or RPLACD; e.g. (SETF (CADR X) Y) operates like (RPLACA (CDR X) Y); c) GET, is inverted by PUTPROP; e.g. (SETF (GET (MUMBLE) BAR) 'LO) works like (PUTPROP (MUMBLE) 'LO BAR) , however, correct order of evaluation is obeyed, namely strict left-to-right from the original form; [as of 12/10/79, the order-of-evaluation specification has a bug in it, but this may be corrected - JONL ] d) PLIST, inverted by SETPLIST, and e) ARRAYCALL inverted by STORE; Macros are expanded 1 step at a time, and a SETF-X property is checked for, in case the user has an overriding expander. If the access is not a SETF-X expansion, or one of the 5 recognized cases, (or a macro or "autoload" which expands into such), then the function +INTERNAL-SETF-X is called on the list of remaining access/value pairs and the result is EVAL'd. The return value is the result of the last form; *note* that the last form is not evaluated twice unless it is safe and cheap to do so. ** NOTE ** On the LISP Machine, only one access/value pair is allowed ** NOTE ** and the return value is not defined. This is a superset of ** NOTE ** the LISP Machine definition. If you wish your code to run ** NOTE ** on the LISP Machine, you will have to limit your usage to ** NOTE ** these constraints, and the similar constrains it places on ** NOTE ** PUSH and POP. In compiled code, no extra overhead is associated with the return value being defined, unless you actually make use of it; the form is analyzed (possibly) differently depending on whether the resultant value is needed. DEFSETF, an autoloadable macro, allows telling SETF, PUSH, and POP how to invert an access; for simple cases, DEFSETF makes accessor inversion definition very easy. (DEFSETF <function> (( () <destructure>) <value>) <retval> <invert-form>) where <function> is the name of the function to invert, <destructure> is a series of symbols to be bound to arguments to the function to be inverted. <value> is the symbol to be bound to the value to be stored. <retval> is T or (), according to whether the return value from the inverted form is the same as <value>. <retval> is a form to CONS up the apropriate code to store into the slot specified. For example, the DEFSETF for CXR: (DEFSETF CXR (( () INDEX HUNK) VALUE) () ;RPLACX returns HUNK, not VALUE `(RPLACX ,INDEX ,HUNK ,VALUE)) Handling LSUBRs and FSUBRs and macros with variable numbers of arguments is not quite as simple. Those interested should see MC:NILCOM;SETF > It is also possible to write other macros which make use of this information in such a way to avoid duplicating forms in the macro in the code, etc. A mechanism to make this easy to use will be provided. 6) As on LISPM, PUSH and POP have been extended to use SETF instead of SETQ. Thus (PUSH 'BAR (CAR (FOO))) is like (SETF (CAR (FOO)) (CONS 'BAR (CAR (FOO))) ** EXCEPT: ** ** NOTE ** Unlike on the LISP Machine, the return value of PUSH is ** NOTE ** defined to be the new value of the list ("stack"). Also, ** NOTE ** there will be no multiple evaluation of the function FOO, ** NOTE ** as would occur on the LISP machine, and the order of evaluation ** NOTE ** is guarenteed to be left-to-right Also, you can do (POP (SAVED-LOCS (GET-FROB)) (CURRENT-LOC (GET-NEW-FROB))) where SAVED-LOCS and CURRENT-LOC are suitable accessor macros to a stack of "LOC"s in a "FROB" and a single "LOC" in a "FROB", respectively. The second argument is optional, and in all cases, the return value is the CAR of the list accessed ** NOTE ** The LISP Machine supports only the 1 argument case, and does ** NOTE ** NOT guarentee single-evaluation of the accessor nor left-right ** NOTE ** order. 7) INCLUDEF is a subr version of INCLUDE, for the sake of transportable code. (INCLUDEF '((NILSRC) DRAMMP >)) is like (INCLUDE ((NILSRC) DRAMMP >)) This allows conditional inclusion, such as according to site or filesystem. 8) A new "Software device" convention for NAMELIST-series functions. Namelists of the form ((LISP) * *) refer to the maclisp system area. I/O functions which accept namelists will look for a PPN property on a symbol before deciding that it is not a device; if found, then that property is substituted for the device-directory field, and the "namelist" tries again. In particular, the symbol LISP, on non-ITS systems, will have an initial PPN property (if in fact there is no "LISP" device), which will link to the directory with the autoload files on it. On the ITS systems, "LISP" has a PPN property of NIL, but the NAMELIST functions have always converted an unknown device name into a directory name. 9) LDB and DPB and other new AUTOLOADable software as fasl file on LISP: files ((LISP) DEFVST FASL), ((LISP) MACAID FASL), ((LISP) LODBYT FASL) DEFVST - a structure-defining package for MACLISP/LISPM/NIL documentation is on LISP;DEFVST DOC MACAID - many "aids" to macro usage. "(HERALD <module> /30)" produces a form which will print out the version number of the <module> when it is being loaded - a number obtained from the second file name of the source when it is being compiled. The reader is referred to commentary in the source file of MACAID for further information. LODBYT - LDB and DPB are LISPM-compatible macros (which in some cases will call the small subroutines *LDB and *DPB). "LOAD-BYTE", a macro with args (<word> <bit-position> <byte-size>) assumes that "<word>" is a 36-bit word, "<bit-position>" is the low-order bit of a byte of length "<byte-size>" bits (0-origin indexing). Bits are numbered from low-order high-order ("right-to-left") with the lowest-order bit being 0. "(DEPOSIT-BYTE <word> <bit-position> <byte-size> <new-byte>)" is similar, but the <new-byte> is inserted into the return copy of the <word>. Although LDB and DPB are documented in the LISP Machine Manual, there is a definition here below for benefit of those who couldn't get a LISPM manual. For LDB and DPB - The representation "<ppss>" assumes that in base 8, this argument is expressing two octal digits "pp" followed by another two octal digits "ss". Then "(LDB <ppss> <word>)" selects a byte of (\ <ppss> 64.) bits in length, which is positioned in <word> with (\ (// <ppss> 64.) 64.) bits to the right of the byte; in other words, <ppss> is a PDP10 byte-pointer right-shifted by 24. bits. For "(DPB <byte> <ppss> <word>)", note that DPB is a non-destructive operation which returns a new value. Some cases of the macro-expansions of LDB and DPB (and LOAD-BYTE and DEPOSITE-BYTE) produce LSH and BOOLE, but others produce calls to the functions *LDB and *DPB; thus, until the compiler begins to open-code *LDB and *DPB, you will need this "macros" file loaded into your run-time environment also. 10) VSAID - VECTOR support package; STRING - support, and NILAID on LISP: NILAID is a NIL-compatibility support package for maclisp. VSAID is a limited support package for using VECTORs and EXTENDs, which is required by DEFVST, STRING and NILAID. STRING has the NIL and LISPM string handling primitives in it, with a few basic ones coded "by hand" for speed. Garbage-collection strategy requires the use of the GC-DAEMON. See the first page of the source file for more information. 11) (STATUS FILESYSTEM-TYPE), (STATUS OPSYSTEM-TYPE), and (STATUS SITE) Warning! (LAST (STATUS FEATURES)) for non-ITS systems will be different. The last item in the "features" list will be (STATUS FILESYSTEM-TYPE) and the next-to-last will be (STATUS SITE); this will mean no change for ITS-supported systems, but will be different on the other kinds of systems. The FILESYSTEM-TYPE is either "ITS", "DEC20", or "DEC10" and is a kind of general "operating-system type". The OPSYSTEM-TYPE is either "TOPS-20" or "TENEX", when the file-system type is DEC20; or, if the file-system type is DEC10, it is among "TOPS-10" (for vanilla-flavoured TOPS-10), "CMU" (for CMU-hacked-up TOPS-10), or "SAIL" (for a SAIL-perverted-WAITS TOPS-10. The SITE name was originally meant to distinguish running on the MIT-AI machine from the MIT-ML machine (but nowadays, the same maclisp system runs unmodified on all ITS machines); nevertheless, the SITE name is among "AI", "ML", "MC", or "DM" for ITS systems, "TOPS-20" or "TENEX" for DEC20 style systems, "SAIL" for (r.i.p) the SU-AI system, and "TOPS-10" or "CMU" for non-SAIL DEC10 style systems. Someday, the DEC10/DEC20 site names may be more informative, e.g. "CMU-10A", "XX", "BBN-D" or whatever. 12) Some things that failed to be documented previously about DEFUN/DEFUN& &WHOLE - DEFMACRO now permits a keyword &WHOLE, to get at whole call form by specifying a variable to be bound to the whole call form. (DEFMACRO FOOM (&WHOLE X) <mumble>) would bind X just like (MACRO FOOM (X) <mumble>) DEFUN reverts to DEFUN& "if necessary"; e.g. for keywords and destructuring Thus in MACLISP, you can use the & keywords, and argument destructuring without expliciting using DEFUN&. For example, (DEFUN FOO ((A . B) &REST W) (LIST A B W)) becomes essentially (DEFUN FOO G0005 (LET ((W (AND (> G0005 1) (LISTIFY (- 1 G0005)))) ((A . B) (ARG 1))) (LIST A B W))) Any required, optional, or rest variable may be null, meaning "ignore". DEFUN& with "&optional" and/or "&rest" keywords (and DEFUNs that are turned into DEFUN& with these keywords) cause the function to become an LEXPR type - (compiled into LSUBR instead of SUBR). Thus, in order to compile other functions calling these functions, you will likely have to give *LEXPR declarations for these functions, even though they are not explicitly of the maclisp LEXPR format. ** DEFUN&-CHECK-ARGS controls whether or not there is run-time checking to see if a DEFUN& with &OPTIONAL or &REST arguments has too many or two few arguments passed. 13) FRETRY, a new function which is in many ways like FRETURN. FRETRY "unwinds" the stack to the point indicated by the first argument, which should be a PDL pointer such as would be in the list returned by EVALFRAME, and it then evaluates the form extracted from its second argument, which should be a list such as would be returned by EVALFRAME. It was intended primarily to be used to "re-try" something that appears to be losing, but which has been corrected (say, in an error break loop). E.g. (SETQ EF (EVALFRAME ...)) (FRETRY (CADR EF) EF) But it could be used more generally, e.g. (SETQ PP (GET-PDL-PTR <some-state>)) (FRETRY PP `(EVAL ,pp (BREAK STOP-AND-LOOK))) In fact, one could define (DEFUN FRETURN (PP X) (SETQ PP (EVALFRAME (1+ PP))) (FRETRY (CADR PP) (CASEQ (CAR PP) (EVAL `(EVAL ,(cadr pp) (QUOTE ,x))) (APPLY `(APPLY ,(cadr pp) (QUOTE (,x))))))) 14) The setting of "+" in the toplevel and break loops modified. If "+" is input to the toplevel or break loops, then the value of "+" will not be set for that time around, thus leaving "+" with a value corresping to the previous input to toplevel or break. 15) SPRIN1, a Pretty-printing version of PRIN1 autoloadable from GRINDEF SPRIN1 is an autoloadable function suitable for assigning to PRIN1, so that the general top-level printer becomes a "SPRINTER". Try, for example, (SETQ PRIN1 'SPRIN1) 16) CLEAR-INPUT and CLEAR-OUTPUT on SFA's now works Hurray! They give it the CLEAR-INPUT and CLEAR-OUTPUT operation, if defined. The argument is always (). 17) USERATOMS-HOOKS, for compiling user-defined "SQUIDified" datatypes This hook provides a means for the user to manipulate user-defined "atomic" datatypes in the compiler, and then substitute a SQUIDform-like piece of code to construct such an object when the form is output to the FASL file. For example, if the user has macros which manipulate strings represented as (STRING-MARKER . <fixnum array>), he would be unable to include any of these strings as literal data in the FASL file, since there is no representation for arrays in FASL files. However, with this hook it would be possible to achieve the effects of outputing these strings directly. The alternative until now has been to use "SQUID" forms, but this has the disadvantage that you cannot then manipualate these things in the same manner as in the interpreter. If the symbol USERATOMS-HOOKS is non-nil, it should be a list of functions of one argument, which will be called on every piece (atomic or not) of any literal object being output to the FASL file. If the object does not need special handling, the function should return (). Otherwise it should return the NCONS of an s-expression to evaluate at load time to create the object. The hook will not be called on an object more than once. EQness is preserved, if two strings or whatever are EQ in the compiler, they will be made EQ in the interpreter (using the atom-table, for you low-level hackers). (eval-when (COMPILE) (defun MYSTRINGHOOK (form) (cond ((atom form) ()) ;If we don't recognize it, return () ((eq (car form) 'STRING-MARKER) ;One of ourse `((CONS 'STRING-MARKER (fillarray (array () ,(arraysize (cdr form))) ',(listarray (cdr form)))))) (t ()))) ;Not one of ours, return (). (push 'MYSTRINGHOOK USERATOMS-HOOKS)) This is not implemented as efficiently as it might be. If it is too slow, it can be speeded up. [Note: the resulting FASL file may not work in LISP 1861 or before, due to a but in FASLOAD] 18) EOF-COMPILE-QUEUE, for queueing forms to be "done" at the end of file. EOF-COMPILE-QUEUE provides a means to "add forms to the end of the file" in the compiler. Thus if you need certain cleanup actions performed at the end of the file, you can push forms onto this queue, which will be compiled in the order they were pushed when the end of the file is reached. It is OK for forms on this queue to push new forms onto the queue.