Tornado Software Architecture

Distribution Support Tools Group
May 5, 1994

Scope

This document contains the Tornado Software Architecture (TSA), which derives from the Tornado Software Architecture Requirements (TSAR), a separate document. Additional documents specify the design of data structures, protocols, and individual subsystems for Tornado.

The TSA describes the set of underlying principles by which a Tornado system is constructed. These principles govern the overall structure of a Tornado system, the behavior of its component subsystems relative to the overall system, and the means of interaction with legacy systems.

Overview

Tornado is a technical architecture designed to replace or supplement the functions of existing business and technical tools and to allow migration toward a common data model and integration with legacy systems and their successors. Tornado allows incremental migration, addition, and expansion of systems. Tornado is neutral with respect to choice of technology for platform, database, human interface, network, and operating system; the ease of integrating a legacy application into a Tornado system depends only slightly upon the base technology of the application.

A Tornado project is a continually evolving design and implementation effort which, over its lifetime, will introduce a number of new tools and connect various new and existing systems. These tools, the tangible result of the development effort, are important to their users. However, none of these tangible tools can be produced without a model of system construction to render the Tornado objectives achievable and provide a basis for defining and understanding the Tornado Software Architecture. This model, called CRISP, establishes a rigid separation of five core functional areas which together support the execution of any group of related systems. These functional areas are: Communication, Representation, Interaction, Storage, and Presentation (hence the acronym).

In Tornado, these five functional areas:

provide transmission media to carry messages between subsystems (Communications),
define a standard format and protocol for all the messages (Representation),
define a means of accessing data for display or computation (Interaction),
provide places to store data permanently (Storage), and
define a way of showing and manipulating the data in its most natural form on a wide range of devices (Presentation).

Additional explanation is given below. All of these areas, except for the Representation part, are flexible; they may be freely altered, added to, removed from, or exchanged during the lifetime of a Tornado system. Compare this to a traditional information system in which each of these areas is a design choice which becomes a permanent, unchangeable feature of the deployed system. In Tornado, the Representation describes the details of today's system -- its data structures, servers, networks, procedures, business rules, etc. -- and goes one step further to specify exactly how any new or changed data will be described in the future. It is this second feature, the Representation part's descriptions of descriptions, which gives Tornado its ability to grow and change with the needs of the business -- not just changes in hardware and software, but also changes in business rules.

The following sections describe the CRISP model in more detail. A system is the combination of all of the parts of a Tornado system, and a subsystem is one of its parts.

Communication

All Tornado subsystems communicate by passing messages through a communication channel. The communication channel may use an actual communications network or pass blocks of data between programs on a single machine.

Representation

All messages have a standard format and interpretation in a Tornado system. The system communicates all data, control, and signalling (e.g. interrupts) using this representation. New or changed subsystems can introduce their own messages without changing older portions of a system.

Interaction

Tornado subsystems access and alter data by communicating values using the standard representation. The same technique is used to request processing and receive results. The location and interpretation of data and services is itself stored as descriptive data. The descriptive data guides access to actual data and services. Changes in the descriptive data take effect immediately without recompiling, reloading, or otherwise regenerating any programs.

Storage

A Tornado system stores data in multiple databases of different kinds; the descriptive data (see Interaction) ties together all the data sources into a uniform view for Tornado. Tornado also allows local copies of data; the design of the data architecture assures that local copies can be updated when the master (reference) copy of the data changes. Tornado can retrieve data from data warehouses for additional system-wide efficiency.

Presentation

A user wants to view information in the form most suitable to her needs and the capabilities of the terminal or workstation. Tornado user-interface programs translate information from its storage form into its presentation form according to the use of the data and the kind of the user's device. Data may carry additional presentation attributes (e.g. colors, icons, styles) for a particular combination of user and device. This way, one user can set up alternate presentations of data, and multiple users can have their own presentation of shared data.

This section has introduced two very important concepts. The first, related to the system's construction, is that Tornado uses a common language -- the Representation -- to pass messages among subsystems. The second concept, related to data organization and its use, is that Tornado uses online descriptive data (see Interaction, above) to actively interpret requests for data and services. These concepts will be reinforced and elaborated throughout this document.

System Architecture

The system architecture specifies a model to follow during the design of the Tornado system data and its subsystem software. The architectural requirements follow from the Tornado Software Architecture Requirements (TSAR) document. The requirements for and design of Tornado subsystems are the subjects of additional documents.

The TSA does not specify formats, standards, or other implementation details -- any instances or suggestions of same in this document are for illustration only.

References to the Tornado Software Architecture Requirements (TSAR) document take the form {3, 4, 7, 8}, where the numbers refer to the numbered objectives in the TSAR.

The Universal Tool

Figure 1 illustrates a universal tool, presented here as a counter example. In this tool, the user accesses an extensible set of functions via a single interface. The functions share data through a common database. Perhaps they also have the ability to communicate with each other through some kind of IPC mechanism. This is a good model from the user's point of view, and representative of good software engineering practice. However, it is less than ideal for a software architecture to be applied to enterprise computing.

    +---------------------------------------------------------------+
    |                                                               |
    |  +---------------------------------------------------------+  |
    |  |                                                         |  |
    |  |                     Human Interface                     |  |
    |  |                                                         |  |
    |  +---------------------------------------------------------+  |
    |      |   ________|___________|___________|_____ ... ___|___   |
    |      |  /        |  /        |  /  IPC   |  /          |  /   |
    |      | /         | /         | /         | /           | /    |
    |  +-------+   +-------+   +-------+   +-------+     +-------+  |
    |  |       |   |       |   |       |   |       |     |       |  |
    |  | f (x) |   | f (x) |   | f (x) |   | f (x) | ... | f (x) |  |
    |  |  1    |   |  2    |   |  3    |   |  4    |     |  n    |  |
    |  +-------+   +-------+   +-------+   +-------+     +-------+  |
    |      |           |           |           |             |      |
    |      |           |           |           |             |      |
    |      |           |           |           |             |      |
    |  +---------------------------------------------------------+  |
    |  |                                                         |  |
    |  |                        Database                         |  |
    |  |                                                         |  |
    |  +---------------------------------------------------------+  |
    |                                                               |
    +---------------------------------------------------------------+

    Figure 1 
    Simplified Do-Everything Tool

This model implicitly suggests a closed-world implementation model. It looks as though there is a single HI, a single database, and a number of plug-in modules that only need to communicate with that particular HI and database. You could certainly build the system architecture around these assumptions, but it would suffer the following problems:

Technology changes. Today's HI and database won't exist (in their current forms, if at all) five years from now.
Economics change. A database, platform, operating system, or other purchased component that makes economic sense today may not be as attractive tomorrow. This is particularly true of top-sellers, due to upgrade costs or escalating license and support costs.
Your needs change. The system you so carefully build around today's combination of platform, OS, network, and database will be particularly difficult to adapt to new components if your architecture depends upon the characteristics of today's hardware and APIs. It's impossible to keep these dependencies out of your architecture unless you start with that goal; portability can't be retrofitted.

The CRISP Model

The TSA is based upon a strict partitioning of implementation concerns, called the CRISP model. CRISP is an acronym for the five core functions: Communication, Representation, Interaction, Storage, and Presentation.

Communication is a channel by which subsystems communicate. Subsystems communicate via a channel even when part of the same program. The TSA specifies, using a standard API, how connections between subsystems are established and broken and how messages are constructed, sent and received. The expression of this API in a specific programming language is specified in a separate document. The API implementation handles the details of providing the API services using a communications protocol such as RPC, TCP/IP, AppleTalk, DECnet, PPP, etc. If the protocol does not provide a reliable communications channel, the API implementation must simulate this through an error detection and recovery (or correction) layer. {13}
Representation is the form in which data, control, and signalling information is communicated between subsystems. Subsystems always communicate using this form whether they are part of the same program, different programs running on one machine, or programs connected via a network. The protocol for message exchange is implemented by the API. A formal syntax defines how to construct a message for communicating with any subsystem, regardless of the content of the message or its intended effect upon the state of the subsystem. This means that a single parser can access any field of any message, regardless of how the content or meaning of messages changes as the overall system evolves. Portions of a message can be related hierarchically by containment; the syntax is recursive. Message fields tagged with unique identifiers support relations whose domain extends across messages. Fields are named and names are unique across all subsystems. Rules, bound to field names via metadata, determine the interpretation of messages. Rules may be arbitrarily complex as well as context-sensitive. {12, 14}
Interaction is the means by which some subsystem accesses, modifies, or manipulates data in a manner appropriate to the task at hand. The description of the data is read at the time the data is referenced; the description provides information about format, interpretation, validation, currency, etc. An Interaction component performs one or both of the following roles:
- a gatekeeper (or perhaps a receptionist) that knows how to interpret requests for data or processing according to the descriptive data
- a service to compute new results from existing data
Any computational process is required to be memoryless; a message requesting some service must always elicit the same result, regardless of any earlier messages processed through the computation. {4}
Storage is where data has a long term, non-volatile residence. The TSA does not specify a particular database model. Formats, relations, and validation of operational data are specified by descriptive data also resident in Storage. A software agent translates between the Representation and the API of a specific database. The TSA allows related data to be distributed in physically or logically separate databases. This allows the business to establish verifiable reference sources of data, and allows a Tornado implementation to make mirror copies of reference data for use on portable computers when network access is unavailable or undesirable. {5, 6}
Presentation is the appearance of data as seen by a user, and the gestures by which the user manipulates data. A user views and manipulates data through display and input devices, respectively. The capabilities of the display and input device, the requirements of the task, and the needs of the user all help to determine the appearance of the data and the gestures used to manipulate the data and display. The TSA does not specify the capabilities of the display and input devices. Equally acceptable devices include a bitmapped graphics workstation with keyboard and mouse, a personal digital assistant with a notepad display and stylus, a character-based terminal, a bar code reader and one-line readout, or a voice input and response system, among others. The TSA allows presentation attributes to be stored as annotations to data, but never as primary data. {3, 15, 16}

The strict separation of these implementation concerns gives the TSA the adaptability needed to handle incremental changes in host platforms, database vendors, enterprise networks, business models, product rules, and personal computers. Therefore, the TSA does not specify any of these components.

A Simple Tornado System

All components of a Tornado system communicate by passing messages over a virtual communications channel (the architecture specifies the logical content of the channel, and not its physical form) organized as a bus (all the messages in the system share the same virtual channel, whether or not the physical implementation uses a bus). {13}

The use of a message bus is very important to managing the complexity of the overall system as new subsystems are introduced. Each Tornado subsystem connects only to the message bus; this involves a (relatively) constant amount of work to connect a new subsystem. On the other hand, a system which is not organized around a message bus must potentially interact with every other system; the Nth subsystem may have to interface with N-1 existing subsystems, all of which may have to change to interact properly with the Nth subsystem. In the general case, N subsystems can have up to

    N (N - 1)
    ---------
        2

interactions. Table 1 shows how the number of interactions grows for both a bus oriented system and a system with N-way communication paths.

    # of Subsystems     N-way Interactions      Bus Interactions
           2                     1                      2
           3                     3                      3
           4                     6                      4
           5                    10                      5
           6                    15                      6
           7                    21                      7
           8                    28                      8
           9                    36                      9
          10                    45                     10
          11                    55                     11
          12                    66                     12
          13                    78                     13
          14                    91                     14
          15                   105                     15
          16                   120                     16
          17                   136                     17
          18                   153                     18
          19                   171                     19
          20                   190                     20                         

    Table 1 
    Interaction Complexity

Even for moderate values of N (4 to 6) the complexity of these N-way interactions is too much for most software organizations to maintain.

The CRISP model extends the simplified model by clearly showing the communications paths between subsystems, and by explicitly condoning the use of multiple components for HI, database, computers, and networks. {1, 2, 13}

The Representation determines the format and protocol of messages in the CRISP model. As shown in Figure 2, the Communications channel carries data in a format specified by the Representation.

    ---- ... --------------------------------------------- ... ----
    ~                                                             ~
    ~        +------------+  +----------------+  +-------+        ~
    ~        |            |  |                |  |       |        ~
    ~   C    |     R      |  |       R        |  |   R   |    C   ~
    ~        |            |  |                |  |       |        ~
    ~        +------------+  +----------------+  +-------+        ~
    ~                                                             ~
    ---- ... --------------------------------------------- ... ----

    Figure 2 
    Information (R) Carried by a Channel (C)

The rest of the figures will abbreviate Figure 2 as ...=======..., labeled R/C (pronounced ``Representation over Communication''). Figure 3 redraws Figure 1 showing the explicit communications. Figure 3 denotes that this entire system (comprised of an HI, several functional modules, and a database) resides on a single machine.

    +---------------------------------------------------------------+
    |                                                               |
    |  Machine                                                      |
    |                                                               |
    |  +---------------------------------------------------------+  |
    |  | P                                                       |  |
    |  |                   Human Interface                       |  |
    |  |                                                         |  |
    |  +---------------------------------------------------------+  |
    |                              |                                |
    |                              |                                |
    |                              |                                |
    | ...=======================================================... |
    |      |           |           |           |     R/C     |      |
    |      |           |           |           |             |      |
    |      |           |           |           |             |      |
    |   I  |        I  |        I  |        I  |          I  |      |
    |  +-------+   +-------+   +-------+   +-------+     +-------+  |
    |  |       |   |       |   |       |   |       |     |       |  |
    |  | data  |   | f (x) |   | f (x) |   | f (x) | ... | f (x) |  |
    |  |access |   |  1    |   |  2    |   |  3    |     |  n    |  |
    |  +-------+   +-------+   +-------+   +-------+     +-------+  |
    |      |           |           |           |             |      |
    |      |           |           |           |             |      |
    |      |           |           |           |             |      |
    |      |           |           |           |     R/C     |      |
    | ...=======================================================... |
    |                              |                                |
    |                              |                                |
    |                              |                                |
    |  +---------------------------------------------------------+  |
    |  | S                                                       |  |
    |  |                        Database                         |  |
    |  |                                                         |  |
    |  +---------------------------------------------------------+  |
    |                                                               |
    +---------------------------------------------------------------+

    Figure 3 
    A Single-Machine Implementation Under the CRISP Model

In Figure 3, it is reasonable to assume that R/C does not involve a networked communications channel. Whatever the mechanism used for C, the messages passed must still be in the R format. This is the most important difference from a procedural program implementation. The fact that the subsystems pass messages to each other, rather than making procedure calls, hides the fact that our single-machine implementation can optimize the message transfers; it could pass pointers to messages in shared memory, or even use direct procedure calls dispatched according to message contents. {13}

Most of the Interaction components provide a computational service -- these are the boxes labeled f(x) in the figure. The remaining Interaction component provides the data access service.

Figure 3 can be rearranged to show the individual subsystems implemented on different machines, communicating via an actual network. This is shown in Figure 4.

    +---------------------------------------------------------------+
    |                                                               |
    |  Machine 1                                                    |
    |                                                               |
    |  +---------------------------------------------------------+  |
    |  | P                                                       |  |
    |  |                   Human Interface                       |  |
    |  |                                                         |  |
    |  +---------------------------------------------------------+  |
    |                              |                                |
    +------------------------------|--------------------------------+
                                   |
                Network            |
      ...=======================================================...
           |           |           |           |     R/C     |
           |           |           |           |             |
    +------|-----------|--------+  |  +--------|-------------|------+
    |      |           |        |  |  |        |             |      |
    |      |           |      M |  |  | M      |             |      |
    |      |           |      a |  |  | a      |             |      |
    |   I  |        I  |      c |  |  | c   I  |          I  |      |
    |  +-------+   +-------+  h |  |  | h  +-------+     +-------+  |
    |  |       |   |       |  i |  |  | i  |       |     |       |  |
    |  | f (x) |   | f (x) |  n |  |  | n  | f (x) | ... | f (x) |  |
    |  |  1    |   |  2    |  e |  |  | e  |  3    |     |  n    |  |
    |  +-------+   +-------+    |  |  |    +-------+     +-------+  |
    |                         2 |  |  | 3                           |
    |                           |  |  |                             |
    +---------------------------+  |  +-----------------------------+
                                   |
    +------------------------------|--------------------------------+
    |                              |                                |
    |  Machine 4    +--------------+---------------+                |
    |               |                              |                |
    |  +------------------------+     +--------------------------+  |
    |  | S                      |     | I                        |  |
    |  |     Database           |     |      Data Access         |  |
    |  |                        |     |                          |  |
    |  +------------------------+     +--------------------------+  |
    |                                                               |
    +---------------------------------------------------------------+

    Figure 4 
    A Multiple-Machine Implementation Under the CRISP Model

In Figure 4, we've shown only one network, so all the components communicate with the network using a common networking protocol. Even this is not necessary; Figure 5 shows how the system can be implemented in its most general form: multiple networks, multiple hosts, and multiple subsystems. We show a different possible deployment of components here -- the Interaction component for data access is shared across the network, and one of the databases uses an agent (shown as [A]) to translate between the Representation and the API of the database (e.g. Oracle). Obviously, the networks must be connected via some kind of device which can translate between protocols; the meaning of R part is unchanged regardless of the underlying network. {2}

    +--------------------+  +---------------------------------------+
    |                    |  |                                       |
    |  Machine 1         |  |  Machine n                            |
    |                    |  |                                       |
    |  +--------------+  |  |  +--------------+   +--------------+  |
    |  |              |  |  |  |              |   |              |  |
    |  | Human Int. 1 |  |  |  | Human Int. 2 |   | Human Int. n |  |
    |  |              |  |  |  |              |   |              |  |
    |  +--------------+  |  |  +--------------+   +--------------+  |
    |   P     |          |  |   P      |           P      |         |
    |         |          |  |          +-------------+----+         |
    |         |          |  |                        |              |
    +---------|----------+  +------------------------|--------------+
              |                                      |
              |  Network 1                           |  Network n
      ...=================...===[Bridge]===...==================...
           |   R/C     |\_________             /     R/C     |
    +------|-----------|--------+ \           |    +---------|------+
    |      |           |        |  |          |    |         |      |
    |      |           |      M |  |          |    | M       |      |
    |      |           |      a |  |          |    | a       |      |
    |   I  |        I  |      c |  |          |    | c    I  |      |
    |  +-------+   +-------+  h |  |          |    | h   +-------+  |
    |  |       |   |       |  i |  |          |    | i   |       |  |
    |  | f (x) |   | f (x) |  n |  |          |    | n   | f (x) |  |
    |  |  1    |   |  2    |  e |  |          |    | e   |  n    |  |
    |  +-------+   +-------+    |  |          |    |     +-------+  |
    |                         2 |  |          |    | 3              |
    |                           |  |          |    |                |
    +---------------------------+  |          |    +----------------+
                                   |          \_____________
    +------------------------------|-----+   +--------------\-------+
    |                              |     |   |              |       |
    |  Machine 4                   |     |   |  Machine 5  [A]      |
    |             +----------------+     |   |              |       |
    |             |                |     |   |   S          |       |
    |  +--------------+   +-----------+  |   |  +----------------+  |
    |  | S            |   | I         |  |   |  |                |  |
    |  |   Database 1 |   |    Data   |  |   |  |  Database n    |  |
    |  |              |   |   Access  |  |   |  |                |  |
    |  +--------------+   +-----------+  |   |  +----------------+  |
    |                                    |   |                      |
    +------------------------------------+   +----------------------+

    Figure 5 
    A General Implementation Under the CRISP Model

Agents

An agent makes the necessary translations between the API on one side and the Representation on the other. An agent might make this translation to connect a database, computational process, or HI to a communications channel using the Representation.

Think of agents as an integration tool rather than as a required part. You will need agents to connect existing systems, tools, or code to a Communications channel. If you're implementing a new subsystem, you may decide instead to make its API be the Representation specified by the CRISP model. Or not -- you may find it easier to build a traditional API and add an Agent later. We treat the agent as an implementation detail and an enabling technology, rather than as a full-fledged part of the architecture. {3}

Figure 6 illustrates several of the ways that agents may be used in a system conforming to the CRISP model. In Case 1, the agent is distinct from the subsystem. The connection between the agent and the subsystem could be a procedural API, a network connection, or something else. Case 2 shows an agent closely coupled to its associated subsystem; in this case the subsystem almost certainly communicates with the agent via a procedural API. Case 3 shows a subsystem communicating directly with the R/C channel; no agent is required. Case 4 is similar to Case 1, but illustrates that an agent could be shared by multiple subsystems. {1, 3}

    +--------+      +--------+      +--------+      +--------+
    |  sub-  |      |  sub-  |      |  sub-  |      |  sub-  |
    | system |      | system |      | system |      | system |
    |        |      |        |      |        |   .  |        |
    |        |      |  [A]   |      |        |   .  |        |
    +--------+      +---|----+      +--------+   .  +--------+
        |               |               |        |      |
        |               |               |        +-----++-...
       [A]              |               |              |
        |               |               |             [A]
        |  R/C          |  R/C          |  R/C         |  R/C
    =======...       ======...       ======...      ======...

      Case 1           Case 2          Case 3         Case 4
     Separate         Integral          No            Shared
      Agent            Agent           Agent           Agent

    Figure 6 
    Possible Agent Arrangements

The physical residence of agents can also vary. Figure 7 shows two possible approaches; there may be others. In Case 1, the agent is separate from the subsystem, but still resident upon the same machine. Here, the agent might be a separate process, and the link from the subsystem might be shared memory, a network socket, a bidirectional byte stream, or some other mechanism provided by the machine's OS. Case 2 shows the agent residing on a machine separate from the subsystems it services. Some networking mechanism transfers messages between the subsystems and the agent. This network is free to use any protocol necessary or available; the agent translates the native protocol to the R/C requirements of the CRISP model. Although not shown in Figure 7, the same network could carry both the native protocol and the R/C format simultaneously; the model does not demand physical separation of the communications paths. {1}

    +--------------+      +--------------+    +--------------+
    | Machine      |      | Machine      |    | Machine      |
    |  +--------+  |      |  +--------+  |    |  +--------+  |
    |  |        |  |      |  |        |  |    |  |        |  |
    |  |  sub-  |  |      |  |  sub-  |  |    |  |  sub-  |  |
    |  | system |  |      |  | system |  |    |  | system |  |
    |  |        |  |      |  |        |  |    |  |        |  |
    |  +--------+  |      |  +--------+  |    |  +--------+  |
    |      |       |      |      |       |    |      |       |
    |      |       |      +------|-------+    +------|-------+
    |     [A]      |             |                   |
    |      |       |         /===========================/
    |      |       |                       |     Network
    +------|-------+                       |    (not R/C)
           |                   +-----------|--+
           |                   | Machine   |  |
    ...=========...            |          [A] |
          R/C                  |           |  |     Case 2
                               +-----------|--+    Different
        Case 1                             |       Machines
     Same Machine                          |
     as Subsystem                   ...=========...

    Figure 7 
    Possible Physical Locations of Agents

Just as agents admit different implementations while still conforming to the TSA, so do other parts of the CRISP model. The appearance of this information in an architectural spec neither constrains the possible implementations nor defines the architecture in terms of possible implementations; rather, we seek to illuminate, by way of example, the succinct architectural definition given early in this document.

Representation

Unlike Communication, Interaction, and Storage, the Representation never has a physical form. Representation is more like a language; in order for different subsystems to communicate effectively, they must all talk the same language. Every message conveyed by the Communication channel must be expressible in the Representation language. The language must be able to express new concepts without ever changing the language itself; otherwise, every time a new kind of message was introduced to the system, every subsystem would have to be rewritten to accept messages in the new language. The TSA does not specify a particular Representation language; this is the subject of System Design. {12}

One possible format for a Representation could be a fully parenthesized nested list syntax combined with an object based slot-value semantics. (See Figure 8 for an example.) Here, ``object based'' means that every data object has a type name, and ``slot-value semantics'' means that the attributes of each object are identified by name. The fully parenthesized notation supports the specification of arbitrary hierarchical relationships between data objects. The object based slot-value semantics ensures that new object types and attributes can be introduced without revising the syntax recognized by existing parsers; such a parser will be able to receive all messages produced by the system now or in the future, but will not need to take any special action to skip over messages that are not relevant to the task of the subsystem served by the parser.

      (#1 (node) 
        (description 
          ((product-info) 
            (product-country 10001 ) 
            (product-base-pc 
              ((product-code) 
                (pc-number 68690 ) ) )
            (product-description "6525" ) 
            (nest-list (
                ((nest-description) 
                  (nest-pc 
                    ((product-code) 
                      (pc-number -92081528 ) ) ) 
                  (slot-info-list (
                      ((slot-entry) 
                        (card-family 6525 ) 
                        (card-pc 
                          ((product-code) 
                            (pc-number 68500 ) ) ) 
                        (card-backplane-pc 
                          ((product-code) 
                            (pc-number 68906 ) ) ) 
                        (card-port-options-list (
                            ((port-options-entry) 
                              (port-option-pc 
                                ((product-code) 
                                  (pc-number 503435 ) ) ) 
                              (port-option-is-cable 0 ) 
                              (port-option-ports (1 2 ) ) ) ) )
                          ) ) )
                  (card-options-list (
                      ((card-option-entry) 
                        (option-pc 
                          ((product-code) 
                            (pc-number 68805 ) ) ) 
                        (option-slot-numbers-list (1 ) ) ) ) 
                    )
                  (power-supply-list (
                      ((power-supply-entry) 
                        (power-supply-pc 
                          ((product-code) 
                            (pc-number 68994 ) ) ) ) ) ) ) ) 
              ) ) ) )

    Figure 8 
    Sample Representation Format

Of course, there are other possible Representation formats that meet the same objectives. For example, a more compact coding could be obtained by mapping the object based slot-value semantics onto a byte stream where the length of a data object is explicitly specified by a count, where names are encoded as binary integers (of length sufficient to ensure an adequate supply of new names for the lifetime of the system), and where values are represented in raw binary rather than translated to ASCII. (What this representation gains in compactness might be offset by the complexity of coding around byte-order issues on different hardware platforms.)

Regardless of the chosen Representation format, the same parser can be used by all subsystems (Figure 9).

    ---- ... --------------------------------------------- ... ----
    ~                                                             ~
    ~        +------------+  +----------------+  +-------+        ~
    ~        |            |  |                |  |       |        ~
    ~   C    |     R      |  |       R        |  |   R   |    C   ~
    ~        |            |  |                |  |       |        ~
    ~        +------------+  +----------------+  +-------+        ~
    ~                     Communications Channel                  ~
    ---- ... --------------------------------------------- ... ----
                 |                                   |
                 |                                   |
                 |                                   |
    +------------|------------+         +------------|------------+
    |            |            |         |            |            |
    |  +-------------------+  |         |  +-------------------+  |
    |  | network interface |  |         |  | network interface |  |
    |  |-------------------|  |         |  |-------------------|  |
    |  |      parser       |<====\  /=====>|      parser       |  |
    |  |-------------------|  |  same   |  |-------------------|  |
    |  |         |         |  |  code   |  |         |         |  |
    |  |  sub-   |   sub-  |  |         |  |  sub-   |   sub-  |  |
    |  | system ... system |  |         |  | system ... system |  |
    |  |   1     |    n    |  |         |  |   1     |    n    |  |
    |  |         |         |  |         |  |         |         |  |
    |  +-------------------+  |         |  +-------------------+  |
    |                         |         |                         |
    | Machine 1               |         | Machine 2               |
    +-------------------------+         +-------------------------+

    Figure 9 
    Shared Code for Representation Parser

Storage

The Storage component always communicates with other subsystems using the Representation language over the Communication channel. This is always true, even when the Storage subsystem has its own communication-based API. Figure 10 illustrates a possible implementation of a Storage component.

    ...============================================================...
                         |      R/C                        |
                         |                                 |
                         |                                 |
    +--------------------|---------------------------------|---------+
    | Machine            |                                 |         |
    |                    |                                 |         |
    |   +---------------------------------+                |         |
    |   |                                 |                |         |
    |   |              Parser             |    +------------------+  |
    |   |---------------------------------|    |                  |  |
    |   |                                 |<<<<|semantics metadata|  |
    |   |            Translator           |    +------------------+  |
    |   |---------------------------------|                          |
    |   |                                 |    +------------------+  |
    |   |          Database Server        |<<<<|                  |  |
    |   +---------------------------------+    | data dictionary  |  |
    |          |                    |          +------------------+  |
    +----------|--------------------|----------------------|---------+
               |                    |                      |
               |        ------------|---------------------/
               |       /            |       /
          ----------  /        ----------  /
         /          \/        /          \/
        /    disk    \       /    disk    \
        \    drive   /       \    drive   /
        |\          /|       |\          /|
        | ---------- |       | ---------- |
        | |        | |  ...  | |        | |
        | |        | |       | |        | |
        \ |        | /       \ |        | /
         \|        |/         \|        |/
          ----------           ----------

    Figure 10 
    An Implementation of a Storage Subsystem

As you might expect, this Storage subsystem contains a database: the disk drives, server software, and data dictionary. Typically the server talks SQL and the data dictionary is maintained by some proprietary tool. The data dictionary describes the data stored in the database -- fields, tables, types, relations, restrictions, etc. A typical (non-CRISP) information system would access the database directly in the format described by its data dictionary. The TSA requires the capability to access multiple databases, each with its own data dictionary, physical location, access methods, and other details. A traditional application would have to include code to deal with each database; adding, removing, or changing a database would require corresponding changes to the program code. The problem is compounded by the introduction of additional systems that need to access these databases. {5}

A system built according to the TSA adds a software layer to translate between the data and control operations of the databases and the subsystems that must access the databases. The translation layer understands the TSA Representation on one side, and the native language of the database on the other side. The important distinction between this approach and a traditional SQL approach is that the TSA subsystems always speak their own language -- the Representation. The benefits are that (1) the Representation language can be matched to the needs of the system, (2) the Representation is independent of evolutionary changes in the actual database systems, and (3) the TSA approach does not require that all data be structured in a relational or tabular format. {1, 5}

The TSA allows related data to be distributed in physically or logically separate databases. Tornado is able to use data wherever it exists; metadata used by the TSA pulls together multiple data sources into one coherent whole that accurately represents the state of the business. Even when the metadata must change to meet changing business needs (for example, changing how data is related or what rules apply), the underlying data remains intact. Traditional systems involve conversions, translations, imports/exports, and other transfers which may produce a loss of information in the transition from one system to its successor or even between versions of the same system; this does not happen in a TSA system. {5, 14}

                  O a             +---------------
                 / \              |     |     |
                /   \             |---------------
             b O ... O c        a |     |b@   |c@
              /     / \           |-------\-----\-
             /     /   \          |     |  \  |  \   +---------------
            O     O ... O         |---------\---  >c |c@   |     |
                 / \     \        |     |    \|      |---------------
                /   \     .       |-----------\---   |     |     |
               O ... O     .      |     |     |\
                    / \     .                   \
                   .   .                         \   +---------------
                  .     .                         >b |b@   |     |
                 .       .                           |--------|------
                                                     |     |  |  |
                                                              .
                                                              .
                                                              .

    Figure 11 
    A Mapping from Hierarchical to Relational Data

Figure 11 suggests how a hierarchical data structure could be mapped onto a relational database. The tree nodes labeled a, b, and c appear as rows in the relational data. Columns in the table containing the a row hold keys b@ and c@ to uniquely identify rows b and c in other tables. In a Tornado system, this kind of mapping would be provided by the Parser and Translator software of Figure 10, under control of metadata also specified and controlled by the system.

    ...=======...    ...=======...    ...=======...    ...=======...
          | R/C            | R/C            | R/C            | R/C
          |                |                |                |
    +-----------+    +-----------+    +-----------+    +-----------+
    |           |    |           |    | Comm Host |    |           |
    | Macintosh |    |    PC     |    |-----------|    |  Unix Box |
    |           |    |           |    | Xlation SW|    |           |
    +-----------+    +-----------+    +-----------+    +-----------+
       |     |          |     |             |             |     |
    keyboard |        printer |            /|\            | ... |
             |                |           . | .           |     |
           mouse            wand         .  |  .     process-1  |
                                            |                   |
                                         modems            process-n
                                        ^   |
                             PDA       ^    /|     VT-220
                            +---+     ^      |     +---+
                            |   |    RF      |     |   |
                            |   |---modem  modem---|   |
                            +---+                  +---+
                                                 |-------|
                                                 \ +++++ /

    Figure 12 
    Some Sample Presentation Components

Interaction

An Interaction component may provide a data access service. It does this by receiving requests for data, looking in the system's descriptive data for location information (i.e. network address of server, access protocol, table name, access key, lookup procedure, validity and currency information, transfer format) and retrieving or storing the data according to its descriptive specifications. {14}

The Interaction component may also respond to requests for computational services, either by accessing such services through descriptive specifications or by providing the service directly.

Presentation

Presentation provides a user with the means to issue processing commands or to select data from the storage subsystems, present it to the user in an appropriate format, and optionally store new or changed data. As you can see from the variety of Presentation components in Figure 12, the needs of the system dictate the method of presentation; the capabilities of the device support the format of the presentation to the user. This approach runs counter to traditional system design, in which presentation is (implicitly or explicitly) an assumed factor in the system architecture; the TSA explicitly requires flexibility in presentation, and generalizes the concept of presentation to include both human and programmatic users. {15}

How can a Macintosh and a character-oriented terminal present the same data to the user? The Macintosh user interface is based on windows, icons, a mouse, and a pointer (the WIMP paradigm). The character terminal can present a set of about 96 characters at fixed locations on a screen, and accepts only keyboard input from the user. We reconcile these differences by defining a Representation format that supports generic graphical attributes (generic, rather than Macintosh-specific, because we may want to support yet another WIMP system at some future time). The Macintosh Presentation software will translate user gestures into commands in the system Representation, and data objects received in that format into screen drawing and other user interface commands. The character terminal will, through a software translation layer, send keyboard commands via the system Representation, and present data objects without their graphical attributes -- perhaps using a forms-based approach rather than the icons presented to the Macintosh user. {15}

Communications

Communications can be provided in different ways depending upon the needs of the system. As Table 2 suggests, this part can be implemented in a layered fashion or buried within a self-contained system; the important aspect is not the exact implementation of the communications substrate, but the fact that the communications channel exists as an entity distinct from the parts implementing the other parts of the CRISP model for a given system. This strict separation of concerns allows the systems integrator to replace one Communications part with another as the supporting infrastructure changes, whether in response to the growth of the system or the replacement of portions of the supporting infrastructure. {13}

    API       Always specific to the implementation
    Logical   AppleTalk, TCP/IP, native IPC, ...
    Physical  10-T, LocalTalk, shared memory, ...

    Table 2 
    Some Possibilities for Communications

Data Architecture

The TSA accommodates the needs of modern corporations by planning for flexibility and change in both the content of the business data and the rules under which the system qualifies and manipulates this data. The TSA also addresses the special problems of preserving the integrity of corporate data when copies are stored on desktop and portable computers. {6, 14}

A traditional information system has only one kind of data: its operational data. The operational data is the data that the overall system manipulates in support of the user's needs. Familiar data objects such as a customer record, a billing record, and an order detail line are all operational data. The traditional information system manipulates this operational data according to rules built into the program code of the system; even systems built using data dictionary tools translate the data dictionary into executable declarations in some programming language -- once the program is finally compiled and installed in the system, the information about the structure and use of the operational data is unchangeable and unknowable without reference to the data dictionary tool and the source language text of the program.

The presence of large numbers of personal computers in a modern information system adds complexity to data administration procedures and simultaneously negates the benefits of a traditional data dictionary by introducing uncontrolled local copies of data. Building personal applications according to a corporate data dictionary has apparent benefits, but these benefits are quickly overwhelmed by several practical problems:

Changes to the corporate data dictionary must be painstakingly coordinated with simultaneous updates to mainframe systems and a potentially large number of personal applications compiled against the data dictionary.
The presence of DBMSs from multiple vendors, each with its own API and data dictionary, each containing some fraction of the total corporate data but lacking mechanisms for automatic coordination with other databases, makes it virtually impossible to create cross-database applications using tools provided by the vendor of any one of the DBMSs without relying upon data extracted from the (relatively) secondary DBMSs. The data extractions introduce the risks identified previously.
Over the lifetime of the corporate information system, the vendor of any DBMS may depart from the desirable path that led to its initial selection. Such departures may include increasingly restrictive or expensive licensing costs, discontinued support of some corporate standard mainframe or desktop machine or a particular revision of operating system, poorer performance or reliability of some subsequent release of the DBMS software, or even the discontinuation of the product line due to failure or merger of the vendor.

The concept of operational data is also part of the TSA -- otherwise the TSA wouldn't be of much use in implementing an information system.

Descriptive Data

To counter the loss (because it is not available online) of information about operational data within an online system, the TSA describes operational data using active, inspectable, modifiable data. This descriptive data is used constantly during the operation of the system; changes in the descriptive data can (if desired) immediately affect the operation of online subsystems. The system stores every piece of descriptive data in a known reference location (just like every piece of operational data); a change to operational or descriptive data in its reference location will eventually be seen and used by all the subsystems. {6, 14}

Descriptive data, implemented according to the TSA, does not replace data dictionaries. Rather, the TSA augments a data dictionary by requiring that its contents be available online, reflecting the current operational state of its database. The descriptive data, much of which can be derived automatically from the contents of a data dictionary, is presented to the system in a format specified according to the needs of the system, rather than the implementation of the DBMS. When multiple DBMSs are involved, this normalized format for descriptive data hides the differences among data dictionaries from different vendors. {5, 14}

The TSA places certain demands on the structure and interpretation of descriptive data, summarized by the following points:

The descriptive data must be designed according to the needs of the system, not the components which support the system.
The format of the descriptive data must be readily extensible to express new concepts.
Extending the format or content of the descriptive data must not require changes to existing subsystems unless the subsystem needs to know about the extended format or content.
A subsystem which is to be replaced by a new subsystem which understands extended format or content of descriptive data must continue to function properly in the presence of the new descriptive data.
The descriptive data, as interpreted by the system, must always refer to the same data object (or an equivalent object) even if the underlying operational data becomes accessible via a different path or is stored in a different internal representation.

How can all this be done? It's probably easiest to begin with something that is easy to visualize but impossible to implement, and then gradually unravel the details of an implementation model. The implementation model described here was designed to meet the needs of the TSA.

Our simple (but unrealizable) model behaves as if every data object (a) has some unique, unchanging name and (b) directly stores or computes the answer to any question we may conceivably want to ask of the object. Remember, this model is the basis of our understanding of the descriptive data in the TSA, not of any implementation!

This gives us everything we need to describe any possible information system -- we can reference data uniquely, and we can get any information related (directly or indirectly) to some object that we know about. Let's start building a usable realization of this idea by expanding upon the second point: if we have a data object, we can get any related information. This gives us (conceptually, if not in fact) containment relationships, classification hierarchies, inheritance, names, values, and attributes. We also know that we can attach code to the data object to compute derived answers. This gives us ordering predicates (for sorting) and all manner of computed results. What's missing is how to get a reference to a particular data object in the first place.

Root Object

You can't assume that you'll know the identity of every data object you may want to use, but you know that you can follow references from any known object to any other object to which it refers (directly or otherwise). Then we can give our system a single object with a well-known identity and require that all the data in the system be accessible by following references from this root object.

The root object is an important concept. It is present, at least in spirit, in every information system. In most systems, the root object is a second-class citizen compared to the operational data; the operational data is stored in files or in named tables in a relational database -- the root object is implicit in the program that binds together all the pieces of data that the system uses. The TSA allows data storage to be reconfigured without affecting the proper operation of all the subsystems; to support this we require the root object to be explicitly accessible in the same manner as all other data in the system. This has two important consequences: (a) the root is manipulable as data, which means that we never have to alter any programs to change references to other objects from the root, and (b) every subsystem needs to know about only one fixed piece of data, namely the location or address of the root.

The requirement for a single root can be satisfied efficiently in a distributed computing environment by storing copies of the root object at multiple locations (e.g. corporate mainframes, departmental servers, and desktop machines) and updating each copy from its reference location as dictated by the design requirements of the system. Note that this implies a mechanism to keep track of reference locations and viability of copied data.

Named Access

If we require that every reference from an object to another object be given a name (for simplicity, we'll consider any array subscripts to be part of the name), then any data object can be located by following a named sequence of references from the root object. This is equivalent to the technique used to identify named files in a file system. For efficiency, we don't require a subsystem to follow a chain of references from the root object for every access.

Iterators

The TSA requires an iterator for every data object; this allows us to retrieve all the directly referenced objects without knowing their names. This provides functionality equivalent to a directory listing function in a file system. Without this, we'd never be able to discover references to objects we didn't already know about -- a condition which is actually acceptable from the viewpoint of design of end-user applications, but not from the broader viewpoint of system maintenance and administration, which will require at least a simple browser application. The iterator also provides system maintainers with a necessary mechanism for transferring data between storage subsystems (by extracting objects from the old databases, then formatting and storing selected objects in the new databases).

Placeholders

The combination of copied data, network access, and portable computers virtually guarantees that some subsystem will be unable at some time to fulfill a user's request for data. The TSA requires that any subsystem which is subject to this possibility be designed to handle the occurrence in the following manner:

Represent the data internally with information that marks it as temporarily unknown. The mark must include the access path to the data. {8}
Optionally allow the user (or the program) to supply a substitute value for missing data. The substitute data must be marked accordingly. The mark must include auditing information (e.g. timestamp, author, and authentication) as appropriate to the needs of the system. {7}
Retain a record of the fact that data is missing or substituted throughout any subsequent calculations that use this data, and keep track of which calculations need to be redone when the actual data becomes available. {7, 8}
Clearly designate on all output or exports any values which are missing or substituted, either directly or through dependence upon values which are missing or substituted. {7, 8}
Attempt to reconcile missing or substituted data according to the circumstances of the occurrence, e.g. periodically if due to an unexpected (assumed transient) failure or upon reconnection to the network if the occurrence was on a disconnected system. {9}
Automatically update reconciled missing values throughout all dependent calculations. {9}
Depending upon the needs of the system, propagate reconciled values of substituted data (with or without notification to the user) or give the user the option of retaining the substitute value. In the latter case, the retained substitute value will be marked as such, and will no longer be considered substituted during future reconciliations. {9, 10}

Portable computers, because they can be operated detached from the corporate network, are at risk of requesting access to unreachable data. Subsystems designed for use on portable computers must either incorporate the above methods of handling unreachable data, or require a live network connection as a prerequisite to operation. The simpler solution of failing to complete the operation, while marginally acceptable for desktop computers with reliable network connections, should never be considered for subsystems which operate on portable computers.

Data Types

In addition to providing data access via named paths, the descriptive data must be capable of representing:

type
aggregation
attributes
type inheritance
containment
relationships
ordering
procedural attachment

For each data object, the system design defines a data type. This type is independent of the form used to store the data object. If needed, the storage subsystem translates between representations.

Aggregation is the grouping together of related data. We make a technical distinction between aggregation and containment (see below). Aggregation defines complex data types, whereas containment is used to represent physical or logical containment of one data object within another. An aggregate structure is composed of slots, each with its own name, data type, contents, and attributes.

Data objects can have attributes that provide additional information about the object. Attributes are not essential to understanding or using the data object. Typically, an attribute describes some aspect of the object's presentation to the user.

We use type inheritance to classify data through specialization of other data types. A new type can inherit the structure of one or more supertypes. The new type can supplement the supertype(s) with additional slots, or can redefine the meaning of existing slots. The system designer must specify rules for conflict resolution when the new type inherits the same slot from multiple supertypes.

Containment describes how one data object is enclosed by another. This enclosure can represent a real-world containment relationship (e.g. some product's CPU module installed in a cabinet) or a logical data relationship (such as a detail line within a customer's order). Containment always implies subordination -- where one piece normally appears within another -- in contrast to aggregation (above) which implies an association of some kind other than enclosure.

Type inheritance and containment define structural relationships in the system's data. The descriptive data should also define named relationships between data objects where these relationships can be helpful to navigate the data. All data must be accessible via a named path from the root object. The requirement for named relationships means that data names must be chosen according to system standards and catalogued in a central location. Association of specialized meanings with particular data names simplifies the navigation of data, and allow the construction of generalized queries.

Data objects may be related to each other according to a lexicographic order. The descriptive data may define an order over a particular type of data object. This ordering can be used by the storage subsystem to define sort keys and indices for efficiency. However, the ordering is only required to define comparison operators for equivalence, predecessor, and successor. If the descriptive data does not define a sorting order for a particular data type, the type is said to be unordered and its order predicates are unavailable.

A slot may have an attached procedure instead of (or in addition to) a value. The procedure may be triggered by read or write access, and may be used to provide a value for its slot upon reading, or to use the value written to the slot (perhaps to check validity, to update dependent values, or to generate an audit entry).

An attached procedure is itself stored as data according to the needs of the system. One possibility is to retrieve the procedure's object code from a library according to the platform upon which the code will execute. Another possibility is to store the procedure in a source text form designed for the system and execute the procedure remotely, and store its access protocol (typically some form of RPC) in lieu of the procedure description for the slot.

Use of descriptive data will promote consistency, timeliness, and accuracy of online information and procedures, which will in turn encourage the construction or mass customization of many small subsystems designed to meet individual users' requirements for function and presentation without compromising underlying corporate data or business rules.

Security

The TSA requires that all data be readable by any user unless explicitly protected. Data can be protected against reading through use of access control mechanisms in the database subsystem. A read failure can produce a signal that the data is protected or provide a substitute value, depending upon the system design and the security requirements for the data. Authorized users must be able to read protected data; authentication information should be transmitted along with each request for data. Authentication information can be transmitted efficiently and securely by creating an access token upon receipt of a full authentication signature (password or other identification), then passing the token in lieu of authentication for the duration of a session. {11}

Concurrency

Whereas data can be read simultaneously by multiple users, it can only be written by a single user. Data must always be written directly to its reference source location; never to a cache. Because data objects may be related to each other, the consistency of these relationships must be maintained during writing. The TSA requires that updates be protected by a transaction. During a transaction, related records are locked for writing, updated individually, and committed all at once. If the transaction fails, no changes are committed. The system design must make appropriate provisions for writers to recover from deadlock and from failed transactions.

Storage

The TSA explicitly avoids specification of storage technologies. These are subject to change in availability and cost-effectiveness over the lifetime of a system; the TSA defines data according to system requirements and requires system implementors to map the system's data structures and relationships onto storage subsystems using intermediate software agents, thus preserving flexibility of choice for the underlying database system components. {5}

Primitive API

Certain functional operations are required to implement a Tornado system. The operations described in this section are primitive operations; more complex operations may be composed of these primitives for clarity or convenience, but a Tornado API may not expose any simpler operations from which the primitives could be composed. The set of primitives expressed in this section is complete -- a system which can not be built using these primitives is not a Tornado system.

The primitive operations are expressed without reference to a programming language syntax. Each operation has a name, parameters, a functional description, a brief note on intended use, and a list of exceptions which may be raised during execution. Part of the system design process is to bind these primitives to functions in a programming language. The bindings need not have identical names or parameter lists in different programming languages, but the function must be equivalent.

Exceptions may be handled according to the capabilities of the programming language and the requirements of the subsystem design. Possibilities include the use of exception handlers (as in Common Lisp or Ada) or function status values (possible in any language).

This set of primitives operates on data structured according to the following containment hierarchy:

    Object
      Slot
        Value
        Object
        Collection
        Attribute
          Value
          Object
          Collection
    Collection
      Object
    Process

Objects, slots, attributes, values, collections, and processes are identified by references which have the property of unique identity. A reference always identifies the same thing, has indefinite lifetime, and may be copied or stored.

The object/slot/attribute distinction provides useful semantic information. An API that specifies only objects loses expressive power. Similarly, while control and signal primitives could be generalized by giving certain slots an associated action, exposing the intention through the API adds expressiveness. Both cases are akin to using data types in a programming language as opposed to treating memory as an undifferentiated sequence of addressable storage locations.

Access paths, names, and searches are described by specifiers. The representation of these is not fixed by the architecture, but by the detailed system design.

    ROOT OBJECT

    Name:        Get Root Object
    Parameters:  (none)
    Results:     reference to root object
    Description: Return a reference to the system's root object.  The
                 reference may be to a cached or local copy so long as
                 it ultimately derives from the one true root.
    Application: This should be the first thing done by a subsystem.
                 There's not much point in continuing if the subsystem
                 can't access the root.
    Exceptions:  insufficient privilege
                 failed to get root object


    REFERENCE PATHS

    Name:        Get Object Using Access Path
    Parameters:  access path specifier
    Results:     reference to specified object
    Description: Return a reference to the object named by the given
                 access path.
    Application: This is the canonical way to get to an object in one
                 step, if you know its name within the system.
    Exceptions:  object does not exist
                 access path specifier is not well formed
                 insufficient privilege
                 object can not be reached
                 
    Name:        Get Object Path From Reference
    Parameters:  reference to an object
    Results:     access path specifier
    Description: Return an access path to the given object.  Where
                 multiple access paths exist to a single object, only
                 one path is returned.
    Application: This is a way to recover an access path specifier
                 from an object reference.
    Exceptions:  object does not exist
                 insufficient privilege
                 can not get object path from reference
                 

    OBJECT CREATION AND DESTRUCTION

    Name:        Create Object
    Parameters:  (none)
    Results:     reference to a new empty object
    Description: Create a reference to a new empty object.  The object
                 reference is guaranteed unique within the system.
                 The object is not attached to any other system data.
    Application: This is the only way to create a new object.
    Exceptions:  couldn't create object
                 
    Name:        Destroy Object
    Parameters:  reference to an object
    Results:     (none)
    Description: Destroy the given object, its slots (but not their
                 referents), and its slots' attributes (but not any
                 referred values).  All references to the object, its
                 slots, and their attributes become invalid.  The
                 system does not attempt to invalidate copied
                 references; the invalidation is detected at time of
                 access.
    Application: This is the only way to destroy an object.
    Exceptions:  insufficient privilege
                 couldn't destroy object
                 

    SLOTS

    Name:        Add Named Slot To Object
    Parameters:  reference to an object
                 slot name specifier
    Results:     reference to an uninitialized slot
    Description: Add a named slot to an existing object.  If the slot
                 is created, it is explicitly marked as uninitialized.
                 If the slot already exists, its value is unaffected
                 (but doing so is an exception condition).
    Application: This is the only way to add content to an object.
    Exceptions:  object doesn't exist
                 slot name specifier is not well formed
                 a slot already exists with this name
                 insufficient privilege
                 couldn't add named slot to object
                 
    Name:        Get Named Slot
    Parameters:  reference to an object
                 slot name specifier
    Results:     reference to the named slot
    Description: Return a reference to a named slot in an object.
    Application: This is the only way to access content in an object.
    Exceptions:  object doesn't exist
                 slot name specifier is not well formed
                 named slot does not exist
                 insufficient privilege
                 couldn't get named slot
                 
    Name:        Remove Slot From Object
    Parameters:  reference to a slot in an object
    Results:     (none)
    Description: Remove a slot from an object.  This destroys any
                 attributes attached to the slot.  References to the
                 slot and its attributes become invalid, but the
                 system does not attempt to invalidate copied
                 references; this invalidation is performed at the
                 time of access.
    Application: This is the only way to remove content from an object.
    Exceptions:  slot doesn't exist
                 insufficient privilege
                 couldn't remove slot
                 
    Name:        Get Slot Value
    Parameters:  reference to a slot
    Results:     reference to slot's value
    Description: Get the value of an existing slot.
    Application: This is the only way to access an object's contents.
    Exceptions:  slot doesn't exist
                 slot is uninitialized
                 insufficient privilege
                 couldn't get slot value
                 
    Name:        Set Slot Value
    Parameters:  reference to a slot
                 reference to a value
    Results:     (none)
    Description: Set the value of an existing slot.  The value can not
                 itself be a slot reference.
    Application: This is the only way to change an object's contents.
    Exceptions:  slot doesn't exist
                 value can't be a slot reference
                 insufficient privilege
                 couldn't set slot value
                 
    Name:        Get Object Slots List
    Parameters:  reference to an object
    Results:     a list of references to slots in the object
    Description: Get a list of slots in an object.  The list may be
                 empty.
    Application: This is the only way to enumerate an object's contents.
    Exceptions:  object doesn't exist
                 insufficient privilege
                 couldn't get object slots list
                 
    Name:        Get Slot Name
    Parameters:  reference to a slot
    Results:     slot name specifier
    Description: Given a slot reference, return the slot's name.
    Application: Use this to get a name for a slot obtained from Get
                 Object Slots List.
    Exceptions:  slot doesn't exist
                 insufficient privilege
                 couldn't get slot name
                 

    ATTRIBUTES

    Name:        Add Named Attribute To Slot
    Parameters:  reference to a slot
                 attribute name specifier
    Results:     reference to the named attribute
    Description: Add a named attribute to a slot.  The attribute is
                 created with the given name and an explicitly
                 uninitialized value.  Note that the returned value is
                 the entire attribute, not just its value.  The
                 attribute's value should be set by passing its
                 reference to Set Attribute Value.
    Application: This is the only way to associate an attribute with a
                 slot.  This creates the attribute, but does not give
                 it a value.
    Exceptions:  slot doesn't exist
                 attribute already exists
                 insufficient privilege 
                 couldn't add named attribute to slot
                 
    Name:        Get Named Attribute From Slot
    Parameters:  reference to a slot
                 attribute name specifier
    Results:     reference to the named attribute
    Description: Get a reference to a slot's named attribute.  If the
                 attribute does not already exist, it is created with
                 the given name and an explicitly uninitialized value.
                 Note that the returned value is the entire attribute,
                 not just its value.  The attribute's value should be
                 set by passing its reference to Set Attribute Value.
    Application: This is the only way to associate an attribute with a
                 slot.  This creates the attribute if needed, but does
                 not give it a value.
    Exceptions:  slot doesn't exist
                 insufficient privilege 
                 couldn't get named attribute from slot
                 
    Name:        Get Attribute Value
    Parameters:  reference to an attribute
    Results:     reference to the attribute's value
    Description: Given a reference to an attribute, extract its value.
    Application: This is the only way to get an attribute's value.
    Exceptions:  attribute doesn't exist
                 insufficient privilege
                 couldn't get attribute value
                 
    Name:        Get Attribute Name
    Parameters:  reference to an attribute
    Results:     reference to the attribute's name
    Description: Given a reference to an attribute, extract its name.
    Application: This is the only way to recover the name of an
                 attribute.
    Exceptions:  attribute doesn't exist
                 insufficient privilege
                 couldn't get attribute name
                 
    Name:        Set Attribute Value
    Parameters:  reference to an attribute
                 reference to attribute value
    Results:     (none)
    Description: Replace any existing value of an attribute.  The
                 reference to an attribute is obtained using Get Named
                 Attribute From Slot.
    Application: The is the only way to set an attribute's value.
    Exceptions:  attribute doesn't exist
                 value doesn't exist
                 insufficient privilege
                 couldn't set attribute value
                 
    Name:        Get Attributes List From Slot
    Parameters:  reference to a slot
    Results:     a list of references to attributes for the slot
    Description: Get a list of all the attributes associated with a
                 given slot.  The list may be empty.
    Application: This is the only way to enumerate all the attributes
                 for a slot.
    Exceptions:  slot doesn't exist
                 slot doesn't have any attributes
                 insufficient privilege
                 couldn't get attributes list from slot
                 
    Name:        Remove Attribute
    Parameters:  reference to an attribute
    Results:     (none)
    Description: Remove an attribute from a slot.
    Application: This is the only way to remove an attribute.
    Exceptions:  attribute doesn't exist
                 insufficient privilege
                 couldn't remove attribute
                 

    COLLECTIONS

    Name:        Create Collection
    Parameters:  (none)
    Results:     reference to a collection
    Description: Create a new collection.  A collection holds an
                 unspecified number of elements (objects).  The
                 collection is created without any elements.  The
                 collection is not attached to any other system data.
    Application: Use a collection to group information which has
                 common characteristics but distinct values.  Examples
                 include the set of all customer records, line items
                 on a purchase order, or rules for configuring a
                 certain product.
    Exceptions:  couldn't create collection
                 
    Name:        Destroy Collection
    Parameters:  reference to a collection
    Results:     (none)
    Description: Destroy the given collection.  Its elements are not
                 destroyed.  All references to the collection become
                 invalid.  The system does not attempt to invalidate
                 copied references; the invalidation is detected at
                 time of access.
    Application: This is the only way to destroy a collection.
    Exceptions:  collection doesn't exist
                 insufficient privilege
                 couldn't destroy collection
                 
    Name:        Add Object To Collection
    Parameters:  reference to a collection
                 reference to an object to add to collection
    Results:     (none)
    Description: Add an object's reference to a collection.  A
                 collection may contain only objects and other
                 collections.
    Application: This is the only way to add an object reference to a
                 collection.
    Exceptions:  collection doesn't exist
                 object is a slot, attribute, or process
                 object doesn't exist
                 insufficient privilege
                 couldn't add element to collection
                 
    Name:        Remove Object From Collection
    Parameters:  reference to a collection
                 reference to an object in the collection
    Results:     (none)
    Description: Remove an object from a collection.  This does not
                 affect the existence of the object or of any
                 references to the object.
    Application: This is the only way to remove an object.
    Exceptions:  collection doesn't exist
                 object doesn't exist
                 object isn't in collection
                 insufficient privilege
                 couldn't remove object from collection
                 
    Name:        Start Collection Enumeration
    Parameters:  reference to a collection
    Results:     enumeration ID
    Description: Return an enumeration ID that can be used with the
                 other collection enumeration functions.
    Application: Use with the other collection enumeration functions
                 to access all the elements in a collection.
    Exceptions:  collection doesn't exist
                 insufficient privilege
                 couldn't start collection enumeration
                 
    Name:        Get Next Collection Object
    Parameters:  enumeration ID
    Results:     reference to an object in the collection
    Description: Returns a reference to the next object in the
                 collection.  The sequence of subsequent objects
                 obtained from Get Next Collection Element must not be
                 affected by passing this reference to Remove Object
                 From Collection.
    Application: This is the only way to get to all the objects in a
                 collection.
    Exceptions:  enumeration ID is not valid
                 no more objects in collection
                 collection was changed by another user
                 insufficient privilege
                 couldn't get next collection object
                 

    TRANSACTIONS

    Name:        Begin Transaction
    Parameters:  (none)
    Results:     transaction ID
    Description: Begin a series of operations in which shared objects
                 will be modified.  Objects to be modified must be
                 flagged by Reserve Object For Update.  All modified
                 objects are committed to their reference source upon
                 execution of Commit Transaction.  If Commit
                 Transaction fails, all objects are left unchanged.
                 Uncommitted modifications can be canceled using
                 Cancel Transaction.  Processes with sufficient
                 privileges may forcibly cancel transactions initiated
                 by other processes.
    Application: This is how we handle concurrent data access.
    Exceptions:  insufficient privilege
                 couldn't begin transaction
                 
    Name:        Commit Transaction
    Parameters:  transaction ID
    Results:     (none)
    Description: see Begin Transaction
    Application: see Begin Transaction
    Exceptions:  transaction doesn't exist
                 transaction was forcibly canceled
                 insufficient privilege
                 couldn't commit transaction
                 
    Name:        Cancel Transaction
    Parameters:  transaction ID
    Results:     (none)
    Description: see Begin Transaction
    Application: see Begin Transaction
    Exceptions:  transaction doesn't exist
                 insufficient privilege
                 couldn't cancel transaction
                 
    Name:        Reserve Object For Update
    Parameters:  reference to an object
                 transaction ID
    Results:     (none)
    Description: see Begin Transaction
    Application: see Begin Transaction
    Exceptions:  object doesn't exist
                 transaction doesn't exist
                 insufficient privilege
                 couldn't reserve object for update
                 
    Name:        Reserve Object For Reading
    Parameters:  reference to an object
                 transaction ID
    Results:     (none)
    Description: Mark an object for reading, so other users can't
                 change the object while it is being read.  Other
                 users can read this object concurrently.
    Application: Use this with the iterators (Get Next Collection
                 Object and Get Next Search Match) if the exception
                 caused by some other user changing the object would
                 be problematic.
    Exceptions:  object doesn't exist
                 transaction doesn't exist
                 insufficient privilege
                 couldn't reserve object for reading
                 
    Name:        List Object Transactions
    Parameters:  reference to an object
    Results:     list of transaction IDs
    Description: Return a list of all transactions currently open on
                 the given object.  The list may be empty.
    Application: A sufficiently privileged process can use this with
                 Cancel Transaction to remove all user reservations on
                 an object.
    Exceptions:  object doesn't exist
                 insufficient privilege
                 can't list object open transactions
                 
    Name:        Get Transaction Details
    Parameters:  transaction ID
    Results:     transaction owner ID
                 list of reserved objects w/ update/read status
    Description: Return the identification of the given transaction's
                 owner, and a list of objects reserved under the
                 transaction.  Each object is flagged according to its
                 reservation type: read or update.
    Application: This may have application in deadlock detection and
                 resolution.
    Exceptions:  transaction doesn't exist
                 insufficient privilege
                 can't get transaction details
                 

    SEARCHES

    Name:        Start Search
    Parameters:  search specifier
    Results:     search ID
    Description: Create a search procedure based upon the search
                 specifier, and return an identifier that can be used
                 to access this procedure.  The search ID is valid
                 only for the life of the process which called Start
                 Search.
    Application: This is the mechanism used to construct a query.
    Exceptions:  insufficient privilege
                 couldn't specify search
                 
    Name:        Get Next Search Match
    Parameters:  search ID
    Results:     reference to found object
    Description: Given a search ID, return the next object which meets
                 the search criteria.
    Application: Use this repeatedly to retrieve query results.
    Exceptions:  not a valid search ID
                 another user changed target data
                 insufficient privilege
                 no more objects to search
                 
    Name:        Reset Search
    Parameters:  search ID
    Results:     (none)
    Description: Reset the given search so the next call to Get Next
                 Search Match will return the first matching object.
    Application: Use this to restart a search if its search procedure
                 was already created by Start Search.
    Exceptions:  not a valid search ID
                 insufficient privilege
                 couldn't reset search
                 

    PROCESSING

    Name:        Connect Process
    Parameters:  process specifier
    Results:     reference to a process
    Description: Return a reference to a specified process.  The
                 process specifier provides all of the information
                 needed to identify the desired process.
    Application: This is the only way to obtain a process to do
                 computation.  All interaction with this process
                 identifies the process using the returned reference.
    Exceptions:  not a valid process specifier
                 process is unavailable for use
                 insufficient privilege
                 couldn't connect process

    Name:        Inquire Process
    Parameters:  reference to a process
    Results:     process status
    Description: Get information about the status of a process.
    Application: Use this to obtain general information about a process.
    Exceptions:  process does not not exist
                 insufficient privilege
                 couldn't inquire process

    Name:        Disconnect Process
    Parameters:  reference to a process
    Results:     (none)
    Description: Invalidate the given process reference and take
                 whatever actions are necessary to release process
                 resources.
    Application: Use this when you won't be sending any more requests
                 to the given process.
    Exceptions:  process does not exist
                 insufficient privilege
                 couldn't disconnect process

    Name:        Request Action
    Parameters:  reference to a process
                 action specifier
    Results:     action ID
    Description: Request that a process initiate a specified action,
                 and return an identifier for the request.  The action
                 specifier includes any needed data.
    Application: Use this to request services from a process.
    Exceptions:  process does not exist
                 invalid action specifier
                 insufficient privilege
                 couldn't request action

    Name:        Inquire Action
    Parameters:  action ID
    Results:     action status
    Description: Return the current status of the given action.
    Application: Use this to discover whether an action has completed
                 or is still in progress.
    Exceptions:  not a valid action ID
                 insufficient privilege
                 couldn't inquire action

    Name:        Request Action Completion Notification
    Parameters:  action ID
    Results:     (none)
    Description: Request a notification signal upon completion of an
                 action.  The notification is transferred via an
                 implementation-dependent interface.
    Application: Use this in lieu of Inquire Action if you'd prefer
                 not to poll for completion.
    Exceptions:  not a valid action ID
                 insufficient privilege
                 couldn't request action completion notification

    Name:        Abort Action
    Parameters:  action ID
    Results:     (none)
    Description: Stop an action that is still in progress.  No results
                 will be available.
    Application: Use this to cancel a service request.
    Exceptions:  not a valid action ID
                 action is not in progress
                 insufficient privilege
                 couldn't abort action

    Name:        Get Action Results
    Parameters:  action ID
    Results:     action results
    Description: Return a (possibly empty) result of the specified
                 action.  The action must have completed.
    Application: Use this to retrieve the results of a service
                 request.
    Exceptions:  not a valid action ID
                 action is still in progress
                 insufficient privileges
                 couldn't get action results

    Name:        Acknowledge Action
    Parameters:  action ID
    Results:     (none)
    Description: Acknowledge completion of the specified action, and
                 invalidate the action ID.
    Application: Call this after you have retrieved any action results.
    Exceptions:  not a valid action ID
                 insufficient privileges
                 couldn't acknowledge action


    SERVICE PROVIDERS

                 Service providers use an implementation-dependent
                 interface compatible with the messages sent by
                 PROCESSING interfaces.


    MISCELLANEOUS

    Name:        
    Parameters:  
    Results:     
    Description: 
    Application: 
    Exceptions:

Sample System-Level Data

This section suggests one possible arrangement of data for a fully-functional system. A typical system design will use most or all of the data described in this section, and add additional data tailored to the requirements of the system design. For clarity, we assume that caches and distributed copies are present in the system -- these aspects are discussed only where the details are important.

In this section, we will describe data access paths as a series of names joined by a < character. The names represent Objects, Slots, Values, Attributes, and Collections. The < character represents containment -- XYZ<PQR means that PQR is contained in XYZ. Later in this section, names are chosen to evoke the meaning of the associated data; a system may choose other names if needed.

All data in a system is accessible from a root object, named ROOT. ROOT is always the leftmost name in a complete data access path. A system may allow specification of partial access paths relative to some basis path which begins at ROOT; the complete pathname is formed by appending the relative path specification to the basis path. For example:

    Basis path:    ROOT<LMN<PQR<TUV
    Relative path: XYZ<ABC
    Complete path: ROOT<LMN<PQR<TUV<XYZ<ABC

The ROOT description, below, is typical of the format we use to describe all of the sample system-level data.

    Name:        ROOT
    Kind:        Object
    Access path: ROOT
    Description: The topmost item in the system's containment
                 hierarchy; all data in the system can be reached by
                 following some access path that starts at ROOT.

Kind is either Object or Collection. As with everything else in this section, the Kind may be different in an actual system implementation. Where appropriate, the description may mention typical Slots, Attributes, and Values; although not shown, the access path for these is constructed according to their containment relationships.

Some data may have more than one access path. For example, a network design may be contained in a collection of all network designs in the system, as well as in the collection of all data used by one or more Systems Engineers, and in the collection of data representing the customer for whom the network is designed. This allows flexible access to data, but is not a requirement of a system design. Where the details of the access path are unimportant, or obvious from context, we may replace portions of the path with an ellipsis.

    Name:        SERVERS
    Kind:        Collection
    Access path: ROOT<SERVERS
    Description: A collection of all the servers in the system.  In
                 this context, servers are generic providers of
                 services -- computation, storage, hardcopy, etc.

    Name:        SERVER
    Kind:        Object
    Access path: ROOT<SERVERS<SERVER
    Slots:       NAME, IP_ADDRESS, SERVICES, ...
    Description: A description of a physical or logical server having
                 a distinct name and IP address and providing one or
                 more services.  The SERVICES slot contains a
                 collection (see SERVICE).

    Name:        SERVICE
    Kind:        Object
    Access path: ROOT<SERVERS<SERVER<SERVICES<SERVICE
    Slots:       NAME, PORT_NUMBER, COST, PERFORMANCE, ...
    Description: A description of a logical service provided by a
                 server.

    Name:        PROCEDURES
    Kind:        Collection
    Access path: ROOT<PROCEDURES
    Description: A collection of all the procedures used by a system.
                 These procedures may encapsulate business knowledge,
                 product rules, report generation, etc.  In addition
                 to a machine-independent specification of each
                 procedure, the collection may store machine-dependent
                 derivations for efficient execution on multiple
                 target machines.

    Name:        PROCEDURE
    Kind:        Object
    Access path: ROOT<PROCEDURES<PROCEDURE
    Slots:       NAME, DESCRIPTION, CODE, ...
    Attributes:  CODE<FORMAT, CODE<OS, CODE<CALLING_SEQUENCE,
                 CODE<LAST_REVISED, DESCRIPTION<CLASSIFICATION, ...
    Description: A description of a single procedure encoded for one
                 particular execution environment.  The procedure is
                 classified according to its purpose to facilitate
                 global searches for procedures suitable to a task.

    Name:        TYPES
    Kind:        Collection
    Access path: ROOT<TYPES
    Description: Descriptions of all objects in the system; how they
                 are composed and what kind of data they contain.

    Name:        TYPE
    Kind:        Object
    Access path: ROOT<TYPES<TYPE
    Slots:       NAME, DESCRIPTION, TEMPLATE, ...
    Attributes:  TEMPLATE<LAST_REVISED, DESCRIPTION<CLASSIFICATION, ...
    Description: A description of a single data type.  The TEMPLATE is
                 a Collection of DESCRIPTORs; taken as a whole, these
                 describe the data type.  Like PROCEDUREs, TYPEs are
                 classified to facilitate search for a type that
                 fulfills a particular role in a system.

    Name:        DESCRIPTOR
    Kind:        Object
    Access path: ROOT<TYPES<TYPE<DESCRIPTOR
    Slots:       NAME, DESCRIPTION, REPRESENTATION, ...
    Attributes:  REPRESENTATION<RANGE, REPRESENTATION<PRINT_FUNCTION,
                 REPRESENTATION<VALIDATION_FUNCTION, ...
    Description: A description of a single data element; not the
                 element itself.  The REPRESENTATION, which is the
                 basis of the type, is taken from a system-wide
                 library.  Depending upon the system design, the
                 representation could be descriptive of a storage
                 format (integer, floating point, counted string,
                 etc.).  Some common attributes are also specified:
                 range, print function, validation, function, etc.

    Name:        USERS
    Kind:        Collection
    Access path: ROOT<USERS
    Description: A collection of all the system users who need to
                 retain some personal association with other data in
                 the system.

    Name:        USER
    Kind:        Object
    Slots:       NAME, DEPARTMENT, TELEPHONE_EXTENSION, ...
    Access path: ROOT<USERS<USER
    Description: The description of a system user.

    Name:        PRODUCTS
    Kind:        Collection
    Access path: ROOT<PRODUCTS
    Description: A collection of descriptions of all the products
                 known to the system.

    Name:        PRODUCT
    Kind:        Object
    Access path: ROOT<PRODUCTS<PRODUCT
    Slots:       NAME, DESCRIPTION, PRODUCT_CODE, FAMILY, BUILD_RULES,
                 VALIDATION_RULES, CONFIGURATION_RULES,
                 PRODUCT_MANAGER, SERVICE_LEVELS, TRANSFER_COST,
                 RETAIL_COST, LEASE_FACTOR, INTRODUCTION_DATE,
                 OBSOLETE_DATE, SUNSET_DATE, ...
    Description: A description of one product.

    Name:        CUSTOMERS
    Kind:        Collection
    Access path: ROOT<CUSTOMERS
    Description: A collection of all customers known to the system.

    Name:        CUSTOMER
    Kind:        Object
    Access path: ROOT<CUSTOMERS<CUSTOMER
                 ROOT<USERS<...<CUSTOMER
    Slots:       NAME, ADDRESS, TELEPHONE, CONTACTS, CLOSED_ORDERS,
                 OPEN_ORDERS, PENDING_PROPOSALS, ...
    Description: A description of a single customer.

    Name:        DESIGNS
    Kind:        Collection
    Access path: ROOT<DESIGNS
    Description: A collection of all the designs known to the system.

    Name:        DESIGN
    Kind:        Object
    Access path: ROOT<DESIGNS<DESIGN
                 ROOT<CUSTOMERS<...<DESIGN
                 ROOT<USERS<...<DESIGN
    Slots:       NAME, DESCRIPTION, ACCOUNT_MANAGER,
                 SYSTEMS_ENGINEERS, REQUIREMENTS, TRAFFIC_PROFILES,
                 PRIOR_ITERATIONS, CURRENT_DESIGN, ...
    Description: A description of a single design.

    Name:        
    Kind:        
    Access path: 
    Description:

References to TSAR

Subsystems
3rd party components
Multiple HI
Memoryless computation
Multiple DB
Reference data
Substitute data
Missing data
Reconcile substitute or missing data
Notification of reconciliation
Security
Self-describing representation
Communications
Descriptive data
Use of presentation attributes
Presentation never primary data

Document History

Created   1994 02 16 by DBL
Draft     1994 03 07 by DBL - initial partitioned model
Draft     1994 03 22 by DBL - separated figures, added data architecture
Draft     1994 03 23 by DBL - edited data architecture; added primitive
                              API skeleton; added builder's guide
Draft     1994 04 01 by DBL - added some detail to primitive API
Draft     1994 04 04 by DBL - added inquiry functions to transaction API;
                              added sample system-level data
Draft     1994 04 13 by DBL - incorporate editorial review comments from
                              JDS, RWS, RGF, and BB; rework for improved
                              organization according to RGF; clarify roles
                              of Interaction, Presentation, and Agent
Draft     1994 04 15 by DBL - rework API section per RGF review; add refs
                              to TSAR
Inspected 1994 04 27-28 by DSTG
Draft     1994 05 05 by DBL - rework per inspection
Converted 1997 03 03 by DBL - converted to HTML