Issue: BACKQUOTE-COMMA-ATSIGN-DOT-COMMA (Version 1)

[Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>]

Michael Greenwald at Symbolics raised this issue on the Common-Lisp
mailing list. I ran this proposal draft by him. He agrees the writeup
expresses the issue he was trying to raise, and he says he has no
preference for which of the two proposals is adopted.  He just wants
us to take some definitive position so he'll know what to implement
for Genera. He asked not to be cc'd in the mail from this point out.
 -kmp

-----
Issue:        BACKQUOTE-COMMA-ATSIGN-DOT-COMMA
Forum:	      Cleanup
References:   Back quote (pp349-351)
Category:     CLARIFICATION/CHANGE
Edit history: 22-Dec-88, Version 1 by Pitman
Status:	      For Internal Discussion

Problem Description:

  The consistent description of backquote has been disrupted by 
  recent changes in the semantics of APPEND.

  The description at the bottom of p350 suggests that
  `(foo ,@bar) can be legitimately interpreted as any of
  #1: (list* 'foo bar)
  #2: (append (list 'foo) bar)
  #3: (append (list 'foo) bar '())
  Unfortunately, issue APPEND-DOTTED has disrupted the equivalences
  here because if BAR holds a dotted list, #1 and #2 are the same (they
  preserve the `dotted cdr'), but #3 is different (it replaces the
  `dotted cdr' with NIL). Under CLtL, a dotted list given to APPEND was
  an error, so this case did not come up. But since ANSI CL will not make
  this an error, this ambiguity must be resolved.  

Proposal (BACKQUOTE-COMMA-ATSIGN-DOT-COMMA:DIVERGENT):

  Define that `(x1 ... xM . ,xN) is equivalent to (LIST* x1 ... xN).
  The actual (EQL) object xN will be used without copying in the result.
  The result itself might not be a proper list (e.g., if xN is an
  atom or dotted list).

  Define that `(x1 ... xM ,@xN) is equivalent to 
  (APPEND (LIST 'x1) ... (list 'xM) xN '()).
  Any top-level list structure of the object xN will be copied in the
  result, replacing (CDR (LAST xN)) with NIL (or replacing xN itself
  with NIL if xN is an atom and issue APPEND-ATOM passes).

Proposal (BACKQUOTE-COMMA-ATSIGN-DOT-COMMA:INTERCHANGEABLE):

  Define that `(x1 ... xM . ,xN) is equivalent to (LIST* x1 ... xN).
  The actual (EQL) object xN will be used without copying in the result.
  The result itself might not be a proper list (e.g., if xN is an
  atom or dotted list).

  Define that `(x1 ... xM ,@xN) is equivalent to 
  (APPEND (LIST 'x1) ... (list 'xM) xN).
  The actual (EQL) object xN will be used without copying in the result.
  The result itself might not be a proper list (e.g., if xN is an
  atom or dotted list).

Notes:

  Note well that this has implications which go beyond dotted lists.
  Currently, `(FOO ,@X) may be implemented by either
     (LIST* 'FOO X)
  or (APPEND (LIST 'FOO) X '())
  or (APPEND (LIST 'FOO) X)
  A consequence of the proposals above is to distinguish between
  the two APPEND cases, forcing changes in the side-effect behavior
  of backquoted structures currently exhibited by some implementations.

Test Cases:

  ;; Issue #1: Non-side-effect treatment of dotted lists.
  (LET ((DOTTED (LIST* 'A 'B 'C)))
    (VALUES `(FOO ,@DOTTED)
	    `(FOO . ,DOTTED)))
  
  => (FOO A B),      (FOO A B . C)		;under proposal DIVERGENT
  => (FOO A B . C),  (FOO A B . C)		;under proposal INTERCHANGEABLE

  ;; Issue #2a: Side-effects
  ;; Sometimes called the ``Standard Backquote Screw''
  ;; Structure is unintentionally shared.
  (LET ((TAIL1 (LIST 'A 'B 'C))
	(TAIL2 (LIST 'A 'B 'C)))
    (FLET ((GET-XABC-COMMA-ATSIGN () `(X ,@TAIL1))
	   (GET-XABC-DOT-COMMA    () `(X . ,TAIL2)))
      (LET ((TEMP1 (GET-XABC-COMMA-ATSIGN))
	    (TEMP2 (GET-XABC-DOT-COMMA)))
	(SETF (CADDR TEMP1) 'Z)
	(SETF (CADDR TEMP2) 'Z)
	(VALUES TEMP1 (GET-XABC-COMMA-ATSIGN)
		TEMP2 (GET-XABC-DOT-COMMA)))))
  => (X A Z C), (X A B C), (X A Z C), (X A Z C) ;under proposal DIVERGENT
  => (X A Z C), (X A Z C), (X A Z C), (X A Z C) ;under proposal INTERCHANGEABLE

  ;; Issue #2b: Side-effects
  ;; Sometimes called ``Inverse Backquote Screw''
  ;; Structure is unintentionally copied.
  (LET ((VAR 'X)
	(VAL '7)
	(THE-SETQ-TAIL (LIST NIL NIL)))
    (LET ((COMMA-ATSIGN `(SETQ ,@THE-SETQ-TAIL))
	  (DOT-COMMA    `(SETQ . ,THE-SETQ-TAIL)))
      (SETF (CAR  SETQ-TAIL) VAR)
      (SETF (CADR SETQ-TAIL) VAL)
      (VALUES COMMA-ATSIGN DOT-COMMA)
  => (SETQ NIL NIL), (SETQ X 7)			; under proposal DIVERGENT
  => (SETQ X 7),     (SETQ X 7)			; under proposal INTERCHANGEABLE

Rationale:

  This clarifies an ambiguity in the description of the language which was caused
  by issue APPEND-DOTTED and APPEND-ATOM.

  Although CLtL tries not to specify the sharing and side-effect implications
  of backquote, there is no really principled reason for its failure to do so.
  In practice, the failure to do so leads to gratuitous portability barriers.
  
Current Practice:

  Currently, the definition of APPEND is such that it would be an error
  to pass it a dotted list, so there is no possibility of discrepancy
  because the interesting case is an "is an error" case.

Cost to Implementors:

  Very small. Some implementations would need to change how they implement
  backquote. Presumably this is a very isolated change.

Cost to Users:

  Technically, none. Existing code is not supposed to rely on the distinctions
  discussed here. The distinction will only be meaningful when ANSI CL goes into
  effect.

  In practice, since some implementations will have to change incompatibly,
  some code which accidentally relies on the current behavior will break.
  However, once such code is fixed, it will be more portable because 
  implementations will not gratuitously diverge.

Cost of Non-Adoption:

  An ambiguity would exist in the language, confusing both users and
  implementors.

Benefits:

  An ambiguity would be removed.
  Some gratuitous barriers to portability would be removed.

Aesthetics:

  Proposal DIVERGENT makes things slightly harder to learn.

  Both proposals make things more predictably portable, which presumably
  has some aesthetic appeal.

Discussion:

  Pitman thinks that either of these choices will be better than the status quo.

  Given that some people already think of ., and ,@ as interchangeable and merely
  a matter of personal style, it would be better to go with option INTERCHANGEABLE.