[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Issue: PATHNAME-LOGICAL (version 2)
- To: CL-Cleanup@sail.stanford.edu
- Subject: Issue: PATHNAME-LOGICAL (version 2)
- From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
- Date: Tue, 23 May 89 13:21 EDT
This issue is on the agenda for the June X3J13 meeting. This issue has
not been written up before. With KMP's help I have prepared a writeup
which we think is ready for release. I'd like to distribute this to
X3J13 as soon as discussion, if any, in the cleanup subcommittee is
completed.
Issue: PATHNAME-LOGICAL
References: Pathnames (pp410-413)
OPEN (p.418), WITH-OPEN-FILE (p.422), RENAME-FILE (p.423),
DELETE-FILE (p.424), PROBE-FILE (p.424),
FILE-WRITE-DATE (p.424), FILE-AUTHOR (p.424), LOAD (p.426),
COMPILE-FILE (p.439), DIRECTORY (p.427), PATHNAME (p.413),
TRUENAME (p.413), MERGE-PATHNAMES (p.415),
MAKE-PATHNAME (p.416), and PARSE-NAMESTRING (p.414).
Related issues: PATHNAME-CANONICAL-TYPE, PATHNAME-COMPONENT-VALUES,
PATHNAME-SUBDIRECTORY-LIST, and PATHNAME-WILD
Category: ADDITION
Edit history: Version 1, 11-May-89, by Moon
Version 2, 18-May-89, by Moon
Problem description:
Pathname values are not portable, but they are sometimes part of a
program, for example the names of files containing the program and the
data used by the program. Moving large programs between sites would
be easier if pathname values did not have to be translated.
Pathname values are nonportable because not all Common Lisp
implementations use the same operating system and file name syntax varies
widely among operating systems. In addition, corresponding files at two
different sites may have different names even when the operating system
is the same; for example, they may be on different directories or
different devices.
The issue of portable pathname values is separate from the issues of
portable pathname operations. See the related issues listed above.
For inter-issue interactions, see the discussion section below.
Proposal (PATHNAME-LOGICAL:ADD):
Define a "logical" file system that looks the same at every site. This
file system is implemented by translating each logical pathname into a
physical pathname on a real file system. The logical pathnames are the
same at all sites, but the translation rules are different at each site,
thus the physical pathnames can be different at each site.
The syntax of a logical pathname namestring is as follows:
host ":" { directory ";" }* [ name ] [ "." type [ "." version ]]
Terminology:
A word consists of one to twelve uppercase letters, digits, and hyphens.
Lowercase letters are translated to uppercase. The consequences of using
other characters, or more than twelve characters, are unspecified. A
wildcard word is "*" (:WILD), which matches anything, or a word with one
or more asterisk characters inserted into it, with no two asterisks
adjacent; each asterisk matches a sequence of zero or more characters.
The maximum number of non-asterisk characters in a wildcard word is 12.
The host is a word that has been defined as a logical pathname host by
using DEFINE-LOGICAL-PATHNAME-TRANSLATIONS.
There is no device, so the device component of a logical pathname is
always :UNSPECIFIC. No other component can be :UNSPECIFIC.
Each directory is a word, a wildcard word, or "**" (:WILD-INFERIORS).
There are no relative directories.
The name is a word or a wildcard word.
The type is a word or a wildcard word.
The version is a positive decimal integer of one to six digits or
"NEWEST" (:NEWEST) or "*" (:WILD). The letters in "NEWEST" can be in
either alphabetic case. The consequences of using anything else as a
version are unspecified.
Some real file systems do not have versions. Logical pathname
translation to such a file system ignores the version. This implies that
a program cannot rely on being able to store more than one version of a
file named by a logical pathname.
The type of a logical pathname for a Common Lisp source file is "LISP".
This should be translated into whatever type is appropriate in a physical
pathname.
The logical pathname host name "SYS" is reserved for the implementation.
The existence and meaning of SYS: logical pathnames is
implementation-defined.
The functions OPEN (and WITH-OPEN-FILE), RENAME-FILE, DELETE-FILE,
PROBE-FILE, FILE-WRITE-DATE, FILE-AUTHOR, LOAD, COMPILE-FILE, DIRECTORY,
and TRUENAME accept logical pathnames and translate them into physical
pathnames. PATHNAME of a stream created by OPEN of a logical pathname is
a logical pathname. TRUENAME, PROBE-FILE, and DIRECTORY never return
logical pathnames. RENAME-FILE with a logical pathname as the second
argument returns a logical pathname as the first value. MERGE-PATHNAMES
returns a logical pathname if and only if its first argument is a logical
pathname or its first argument does not specify a host and the default
host is logical. MAKE-PATHNAME returns a logical pathname if and only if
the host is logical. PARSE-NAMESTRING returns a logical pathname if and
only if the string begins with a logical pathname host name (defined by
using DEFINE-LOGICAL-PATHNAME-TRANSLATIONS) and a colon, or the string
does not specify a host and the default host is logical.
Add these defined names to Common Lisp in support of logical pathnames:
LOGICAL-PATHNAME [Class]
LOGICAL-PATHNAME is a subclass of PATHNAME.
TRANSLATE-LOGICAL-PATHNAME pathname [Function]
Translate a logical pathname to the corresponding physical pathname.
The pathname argument is first coerced to a pathname. If it is not a
pathname, string, or file stream an error of type TYPE-ERROR is
signalled. If the coerced argument is a logical pathname, the first
matching translation (according to PATHNAME-MATCH-P) of the logical
pathname host is applied, using TRANSLATE-PATHNAME with the reversible
argument true, and three values are returned:
1. The physical pathname
2. The from-wildcard of the translation
3. The to-wildcard of the translation
If no translation matches, an error of type FILE-ERROR is signalled.
If the coerced argument is a physical pathname, it is returned as all
three values.
DEFINE-LOGICAL-PATHNAME-TRANSLATIONS host translations &key [Function]
Define a logical pathname host named <host> (a string or a symbol which
is coerced to a string). <translations> is a list of translations.
Each translation is a list of from-wildcard and to-wildcard.
From-wildcard must be a logical pathname or a string coercible to a
logical pathname. To-wildcard must be a physical pathname or a string
coercible to a physical pathname. Translations are searched in the
order listed, so more specific from-wildcards must precede more general
ones.
The specified translations might be modified or augmented in an
implementation-dependent fashion, typically to provide translation of
file types to local naming conventions, to accomodate physical file
systems with limited length names, or to deal with special character
requirements such as translating hyphens to underscores or uppercase
letters to lowercase. These modifications are reflected in the second
and third values returned by TRANSLATE-LOGICAL-PATHNAME in such a way
that TRANSLATE-PATHNAME used with them produces the same translation.
If a logical pathname host named <host> already exists, its existing
translations are replaced.
There are no keyword arguments specified by this standard, but any
implementation extensions are provided as keyword arguments or as
translations with more than two elements.
LOAD-LOGICAL-PATHNAME-TRANSLATIONS host [Function]
If a logical pathname host named <host> (a string or a symbol which is
coerced to a string) is already defined, return NIL. Otherwise, search
for a logical pathname host definition in an implementation defined
manner. If none is found, signal an error. If a definition is found,
install it and return T.
The search used by LOAD-LOGICAL-PATHNAME-TRANSLATIONS should be
documented, as logical pathname definitions will be created by users,
not only by Lisp implementors.
COMPILE-FILE-PATHNAME pathname &key :output-file [Function]
Returns the pathname that COMPILE-FILE would write into, if given
the same arguments.
Examples:
;A simple, and typical, example
(define-logical-pathname-translations "FOO"
'(("FOO:**;*.*.*" "MY-LISPM:>library>foo>**>")))
;A more complex example
(define-logical-pathname-translations "PROG"
'(("PROG:RELEASED;*.*.*" "MY-UNIX:/sys/bin/my-prog/")
("PROG:RELEASED;*;*.*.*" "MY-UNIX:/sys/bin/my-prog/*/")
("PROG:EXPERIMENTAL;*.*.*" "MY-UNIX:/usr/Joe/development/prog/")
("PROG:EXPERIMENTAL;DOCUMENTATION;*.*.*"
"MY-VAX:SYS$DISK:[JOE.DOC]")
("PROG:EXPERIMENTAL;*;*.*.*" "MY-UNIX:/usr/Joe/development/prog/*/")
("PROG:MAIL;**;*.MAIL" "MY-VAX:SYS$DISK:[JOE.MAIL.PROG...]*.MBX")))
;This function is like DIRECTORY, but if its argument is a logical
;pathname it returns logical pathnames in the results. If its
;argument is a physical pathname, it is the same as DIRECTORY.
(defun logical-directory (pathname)
(multiple-value-bind (physical from-wildcard to-wildcard)
(translate-logical-pathname pathname)
(map 'list #'(lambda (truename)
(translate-pathname truename to-wildcard
from-wildcard t))
(directory physical))))
Rationale:
Large programs can be moved between sites without changing any pathnames,
provided all pathnames used are logical. A portable system construction
tool can be created that operates on programs defined as sets of files.
Logical pathname syntax was chosen to be easily translated into most
popular file systems, while still being powerful enough for storing large
programs. Logical pathnames have least-common-denominator capabilities.
Although they have hierarchical directories, versions, and a medium sized
maximum name length, they can be mapped onto a less capable real file
file system by translating each directory that is used into a flat
directory name, treating all versions as :newest, and/or using special
implementation-dependent translation rules to shorten long names.
More advanced capabilities such as relative pathnames and a name for the
root directory were not felt to be necessary in logical pathnames. They
could be added later if a need emerges.
It is not a goal of logical pathnames to be able to represent all
possible file names. Their goal is rather to represent just enough file
names to be useful for storing software. Real pathnames, in contrast,
need to provide a uniform interface to all possible file names, including
names and naming conventions that are not under the control of Common
Lisp.
The choice of logical pathname syntax was guided by the goal of being
visually distinct from real file systems.
The numbers twelve and six are arbitrary but chosen to accomodate
both typical program file names and typical file system limitations.
File systems that are more limited can still be accomodated through
additional implementation-dependent translation as pointed out earlier.
The LOGICAL-PATHNAME class exists so that methods can distinguish
logical pathnames from regular pathnames.
The two extra values returned by TRANSLATE-LOGICAL-PATHNAME allow
for back-translation, as shown in the LOGICAL-DIRECTORY example.
Loading of logical pathname translations from a site-dependent file
allows software to be distributed using logical pathnames. The software
is supplied with logical pathnames and a sample set of translations. The
actual translations are defined by the user of the software, since the
supplier does not know the user's local file system conventions. Loading
the software uses these translations via LOAD-LOGICAL-PATHNAME-TRANSLATIONS.
The COMPILE-FILE-PATHNAME function and the specification of "LISP" as the
type of a logical pathname for a Common Lisp source file together provide
enough information about compilation for a portable system construction
tool that uses logical pathnames to work.
Current practice:
Symbolics Genera has had a similar facility for many years. It is used
extensively for software distribution by Symbolics and its customers.
The Genera facility uses the same logical pathname syntax but different
function names, and is somewhat more complicated. The extra complexity
is not necessary in the Common Lisp standard.
Symbolics Genera offers a function for translating from a physical
pathname back to a logical pathname. There are a number of problems with
this, and so it has not been proposed here. Instead
TRANSLATE-LOGICAL-PATHNAME returns enough information to allow the user
program to perform the backtranslation itself.
The Genera equivalent of LOAD-LOGICAL-PATHNAME-TRANSLATIONS looks for
a file named SYS:SITE;hostname.TRANSLATIONS.
Cost to Implementors:
This is a fairly complex facility, but its performance is unimportant
so a straightforward implementation should suffice. Most of the
complexity comes in dealing with unusual file systems, such as ones
that don't allow file names longer than eight characters.
Cost to Users:
None.
Cost of non-adoption:
Portable software construction and distribution will have to rely on
implementation-dependent kludges. Lisp software will continue to be
difficult to install.
Performance impact:
None.
Benefits:
Avoid cost of non-adoption.
Esthetics:
Improved portability of large programs.
Discussion:
Issue PATHNAME-LOGICAL fundamentally depends on issue PATHNAME-WILD.
If PATHNAME-CANONICAL-TYPE:NEW-CONCEPT passes, it will affect the
behavior of the function TRANSLATE-PATHNAME and therefore the behavior of
the function TRANSLATE-LOGICAL-PATHNAME. When a logical pathname
translation has from and to type fields that are * or omitted,
translation of the type will be guided by canonical types. If
PATHNAME-CANONICAL-TYPE:NEW-CONCEPT fails to pass, it will either have to
be done behind the scenes by TRANSLATE-PATHNAME or users will have to
write more verbose translations that individually specify the handling of
each file type.