[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Issue: PATHNAME-LOGICAL (version 2)



This issue is on the agenda for the June X3J13 meeting.  This issue has
not been written up before.  With KMP's help I have prepared a writeup
which we think is ready for release.  I'd like to distribute this to
X3J13 as soon as discussion, if any, in the cleanup subcommittee is
completed.

Issue:          PATHNAME-LOGICAL
References:     Pathnames (pp410-413)
                OPEN (p.418), WITH-OPEN-FILE (p.422), RENAME-FILE (p.423),
                DELETE-FILE (p.424), PROBE-FILE (p.424),
                FILE-WRITE-DATE (p.424), FILE-AUTHOR (p.424), LOAD (p.426),
                COMPILE-FILE (p.439), DIRECTORY (p.427), PATHNAME (p.413),
                TRUENAME (p.413), MERGE-PATHNAMES (p.415), 
                MAKE-PATHNAME (p.416), and PARSE-NAMESTRING (p.414).
Related issues: PATHNAME-CANONICAL-TYPE, PATHNAME-COMPONENT-VALUES, 
                PATHNAME-SUBDIRECTORY-LIST, and PATHNAME-WILD
Category:       ADDITION
Edit history:   Version 1, 11-May-89, by Moon
                Version 2, 18-May-89, by Moon

Problem description:

  Pathname values are not portable, but they are sometimes part of a
  program, for example the names of files containing the program and the
  data used by the program.  Moving large programs between sites would
  be easier if pathname values did not have to be translated.

  Pathname values are nonportable because not all Common Lisp
  implementations use the same operating system and file name syntax varies
  widely among operating systems.  In addition, corresponding files at two
  different sites may have different names even when the operating system
  is the same; for example, they may be on different directories or
  different devices.

  The issue of portable pathname values is separate from the issues of
  portable pathname operations.  See the related issues listed above.
  For inter-issue interactions, see the discussion section below.

Proposal (PATHNAME-LOGICAL:ADD):

  Define a "logical" file system that looks the same at every site.  This
  file system is implemented by translating each logical pathname into a
  physical pathname on a real file system.  The logical pathnames are the
  same at all sites, but the translation rules are different at each site,
  thus the physical pathnames can be different at each site.

  The syntax of a logical pathname namestring is as follows:

     host ":" { directory ";" }* [ name ] [ "." type [ "." version ]]

  Terminology:
  A word consists of one to twelve uppercase letters, digits, and hyphens.
  Lowercase letters are translated to uppercase.  The consequences of using
  other characters, or more than twelve characters, are unspecified.  A
  wildcard word is "*" (:WILD), which matches anything, or a word with one
  or more asterisk characters inserted into it, with no two asterisks
  adjacent; each asterisk matches a sequence of zero or more characters.
  The maximum number of non-asterisk characters in a wildcard word is 12.

  The host is a word that has been defined as a logical pathname host by
  using DEFINE-LOGICAL-PATHNAME-TRANSLATIONS.

  There is no device, so the device component of a logical pathname is
  always :UNSPECIFIC.  No other component can be :UNSPECIFIC.

  Each directory is a word, a wildcard word, or "**" (:WILD-INFERIORS).
  There are no relative directories.

  The name is a word or a wildcard word.

  The type is a word or a wildcard word.

  The version is a positive decimal integer of one to six digits or
  "NEWEST" (:NEWEST) or "*" (:WILD).  The letters in "NEWEST" can be in
  either alphabetic case.  The consequences of using anything else as a
  version are unspecified.

  Some real file systems do not have versions.  Logical pathname
  translation to such a file system ignores the version.  This implies that
  a program cannot rely on being able to store more than one version of a
  file named by a logical pathname.

  The type of a logical pathname for a Common Lisp source file is "LISP".
  This should be translated into whatever type is appropriate in a physical
  pathname.

  The logical pathname host name "SYS" is reserved for the implementation.
  The existence and meaning of SYS: logical pathnames is
  implementation-defined.

  The functions OPEN (and WITH-OPEN-FILE), RENAME-FILE, DELETE-FILE,
  PROBE-FILE, FILE-WRITE-DATE, FILE-AUTHOR, LOAD, COMPILE-FILE, DIRECTORY,
  and TRUENAME accept logical pathnames and translate them into physical
  pathnames.  PATHNAME of a stream created by OPEN of a logical pathname is
  a logical pathname.  TRUENAME, PROBE-FILE, and DIRECTORY never return
  logical pathnames.  RENAME-FILE with a logical pathname as the second
  argument returns a logical pathname as the first value.  MERGE-PATHNAMES
  returns a logical pathname if and only if its first argument is a logical
  pathname or its first argument does not specify a host and the default
  host is logical.  MAKE-PATHNAME returns a logical pathname if and only if
  the host is logical.  PARSE-NAMESTRING returns a logical pathname if and
  only if the string begins with a logical pathname host name (defined by
  using DEFINE-LOGICAL-PATHNAME-TRANSLATIONS) and a colon, or the string
  does not specify a host and the default host is logical.

  Add these defined names to Common Lisp in support of logical pathnames:

  LOGICAL-PATHNAME                                              [Class]

    LOGICAL-PATHNAME is a subclass of PATHNAME.

  TRANSLATE-LOGICAL-PATHNAME pathname                           [Function]

    Translate a logical pathname to the corresponding physical pathname.
    The pathname argument is first coerced to a pathname.  If it is not a
    pathname, string, or file stream an error of type TYPE-ERROR is
    signalled.  If the coerced argument is a logical pathname, the first
    matching translation (according to PATHNAME-MATCH-P) of the logical
    pathname host is applied, using TRANSLATE-PATHNAME with the reversible
    argument true, and three values are returned:
      1. The physical pathname
      2. The from-wildcard of the translation
      3. The to-wildcard of the translation
    If no translation matches, an error of type FILE-ERROR is signalled.
    If the coerced argument is a physical pathname, it is returned as all
    three values.

  DEFINE-LOGICAL-PATHNAME-TRANSLATIONS host translations &key   [Function]

    Define a logical pathname host named <host> (a string or a symbol which
    is coerced to a string).  <translations> is a list of translations.
    Each translation is a list of from-wildcard and to-wildcard.
    From-wildcard must be a logical pathname or a string coercible to a
    logical pathname.  To-wildcard must be a physical pathname or a string
    coercible to a physical pathname.  Translations are searched in the
    order listed, so more specific from-wildcards must precede more general
    ones.

    The specified translations might be modified or augmented in an
    implementation-dependent fashion, typically to provide translation of
    file types to local naming conventions, to accomodate physical file
    systems with limited length names, or to deal with special character
    requirements such as translating hyphens to underscores or uppercase
    letters to lowercase.  These modifications are reflected in the second
    and third values returned by TRANSLATE-LOGICAL-PATHNAME in such a way
    that TRANSLATE-PATHNAME used with them produces the same translation.

    If a logical pathname host named <host> already exists, its existing
    translations are replaced.

    There are no keyword arguments specified by this standard, but any
    implementation extensions are provided as keyword arguments or as
    translations with more than two elements.

  LOAD-LOGICAL-PATHNAME-TRANSLATIONS host                       [Function]

    If a logical pathname host named <host> (a string or a symbol which is
    coerced to a string) is already defined, return NIL.  Otherwise, search
    for a logical pathname host definition in an implementation defined
    manner.  If none is found, signal an error.  If a definition is found,
    install it and return T.

    The search used by LOAD-LOGICAL-PATHNAME-TRANSLATIONS should be
    documented, as logical pathname definitions will be created by users,
    not only by Lisp implementors.

  COMPILE-FILE-PATHNAME pathname &key :output-file              [Function]

    Returns the pathname that COMPILE-FILE would write into, if given
    the same arguments.

Examples:

  ;A simple, and typical, example
  (define-logical-pathname-translations "FOO"
     '(("FOO:**;*.*.*"              "MY-LISPM:>library>foo>**>")))

  ;A more complex example
  (define-logical-pathname-translations "PROG"
     '(("PROG:RELEASED;*.*.*"       "MY-UNIX:/sys/bin/my-prog/")
       ("PROG:RELEASED;*;*.*.*"     "MY-UNIX:/sys/bin/my-prog/*/")
       ("PROG:EXPERIMENTAL;*.*.*"   "MY-UNIX:/usr/Joe/development/prog/")
       ("PROG:EXPERIMENTAL;DOCUMENTATION;*.*.*"
                                    "MY-VAX:SYS$DISK:[JOE.DOC]")
       ("PROG:EXPERIMENTAL;*;*.*.*" "MY-UNIX:/usr/Joe/development/prog/*/")
       ("PROG:MAIL;**;*.MAIL"       "MY-VAX:SYS$DISK:[JOE.MAIL.PROG...]*.MBX")))

  ;This function is like DIRECTORY, but if its argument is a logical
  ;pathname it returns logical pathnames in the results.  If its
  ;argument is a physical pathname, it is the same as DIRECTORY.
  (defun logical-directory (pathname)
    (multiple-value-bind (physical from-wildcard to-wildcard)
        (translate-logical-pathname pathname)
      (map 'list #'(lambda (truename)
                     (translate-pathname truename to-wildcard
                                         from-wildcard t))
           (directory physical))))

Rationale:

  Large programs can be moved between sites without changing any pathnames,
  provided all pathnames used are logical.  A portable system construction
  tool can be created that operates on programs defined as sets of files.

  Logical pathname syntax was chosen to be easily translated into most
  popular file systems, while still being powerful enough for storing large
  programs.  Logical pathnames have least-common-denominator capabilities.
  Although they have hierarchical directories, versions, and a medium sized
  maximum name length, they can be mapped onto a less capable real file
  file system by translating each directory that is used into a flat
  directory name, treating all versions as :newest, and/or using special
  implementation-dependent translation rules to shorten long names.
  More advanced capabilities such as relative pathnames and a name for the
  root directory were not felt to be necessary in logical pathnames.  They
  could be added later if a need emerges.

  It is not a goal of logical pathnames to be able to represent all
  possible file names.  Their goal is rather to represent just enough file
  names to be useful for storing software.  Real pathnames, in contrast,
  need to provide a uniform interface to all possible file names, including
  names and naming conventions that are not under the control of Common
  Lisp.

  The choice of logical pathname syntax was guided by the goal of being
  visually distinct from real file systems.

  The numbers twelve and six are arbitrary but chosen to accomodate
  both typical program file names and typical file system limitations.
  File systems that are more limited can still be accomodated through
  additional implementation-dependent translation as pointed out earlier.

  The LOGICAL-PATHNAME class exists so that methods can distinguish
  logical pathnames from regular pathnames.

  The two extra values returned by TRANSLATE-LOGICAL-PATHNAME allow
  for back-translation, as shown in the LOGICAL-DIRECTORY example.

  Loading of logical pathname translations from a site-dependent file
  allows software to be distributed using logical pathnames.  The software
  is supplied with logical pathnames and a sample set of translations.  The
  actual translations are defined by the user of the software, since the
  supplier does not know the user's local file system conventions.  Loading
  the software uses these translations via LOAD-LOGICAL-PATHNAME-TRANSLATIONS.

  The COMPILE-FILE-PATHNAME function and the specification of "LISP" as the
  type of a logical pathname for a Common Lisp source file together provide
  enough information about compilation for a portable system construction
  tool that uses logical pathnames to work.

Current practice:

  Symbolics Genera has had a similar facility for many years.  It is used
  extensively for software distribution by Symbolics and its customers.
  The Genera facility uses the same logical pathname syntax but different
  function names, and is somewhat more complicated.  The extra complexity
  is not necessary in the Common Lisp standard.

  Symbolics Genera offers a function for translating from a physical
  pathname back to a logical pathname.  There are a number of problems with
  this, and so it has not been proposed here.  Instead
  TRANSLATE-LOGICAL-PATHNAME returns enough information to allow the user
  program to perform the backtranslation itself.

  The Genera equivalent of LOAD-LOGICAL-PATHNAME-TRANSLATIONS looks for
  a file named SYS:SITE;hostname.TRANSLATIONS.

Cost to Implementors:

  This is a fairly complex facility, but its performance is unimportant
  so a straightforward implementation should suffice.  Most of the
  complexity comes in dealing with unusual file systems, such as ones
  that don't allow file names longer than eight characters.

Cost to Users:

  None.

Cost of non-adoption:

  Portable software construction and distribution will have to rely on
  implementation-dependent kludges.  Lisp software will continue to be
  difficult to install.

Performance impact:

  None.

Benefits:

  Avoid cost of non-adoption.

Esthetics:

  Improved portability of large programs.

Discussion:

  Issue PATHNAME-LOGICAL fundamentally depends on issue PATHNAME-WILD.

  If PATHNAME-CANONICAL-TYPE:NEW-CONCEPT passes, it will affect the
  behavior of the function TRANSLATE-PATHNAME and therefore the behavior of
  the function TRANSLATE-LOGICAL-PATHNAME.  When a logical pathname
  translation has from and to type fields that are * or omitted,
  translation of the type will be guided by canonical types.  If
  PATHNAME-CANONICAL-TYPE:NEW-CONCEPT fails to pass, it will either have to
  be done behind the scenes by TRANSLATE-PATHNAME or users will have to
  write more verbose translations that individually specify the handling of
  each file type.