6.1 TECHNICAL LEVELS OF INTERACTION BETWEEN OAIS ARCHIVES: Difference between revisions

From wiki.dpconline.org
Jump to navigation Jump to search
mNo edit summary
No edit summary
Line 44: Line 44:




6.1.3 FEDERATED ARCHIVES
 
== 6.1.3 FEDERATED ARCHIVES ==
 


Federated Archives are conceptually Consumer-oriented. In addition to the Local Community (i.e., the original Designated Community served by the Archive), there exists a Global Community (i.e., an extended Designated Community) which has interests in the holdings of several OAIS Archives and has influenced those Archives to provide access to their holdings via one or more common finding aids. However, the Local Consumers are likely to have access priority over the global Consumers.
Federated Archives are conceptually Consumer-oriented. In addition to the Local Community (i.e., the original Designated Community served by the Archive), there exists a Global Community (i.e., an extended Designated Community) which has interests in the holdings of several OAIS Archives and has influenced those Archives to provide access to their holdings via one or more common finding aids. However, the Local Consumers are likely to have access priority over the global Consumers.

Revision as of 11:53, 13 August 2015

In general one OAIS is not interoperable with another; however, there are a number of reasons that some level of interoperability may be desirable, motivated for example by Users, Producers or Management; even interoperable Archives may have different Designated Communities—even for the same digital objects—and hence different requirements for Representation Information and/or Descriptive Information.

OAIS associations can be categorized technically by both external and internal factors. External factors include characteristics of the Producer and Consumer communities. Internal factors could include common implementations of the information models presented in 4.2, or multi-Archive sharing of one or more of the functional areas presented in 4.1.

This subsection defines four categories of Archive association. The first three categories have successively higher degrees of interaction:

Independent: Archives motivated by local concerns with no management or technical interaction among them.

Cooperating: Archives with potential common producers, common submission standards, and common dissemination standards, but no common finding aids.

Federated: Archives with both a Local Community (i.e., the original Designated Community served by the Archive) and a Global Community (i.e., an extended Designated Community) which has interests in the holdings of several OAIS Archives and has influenced those Archives to provide access to their holdings via one or more common finding aids. The Access needs of the Local Community usually have priority over those of the Global Community. Global dissemination and Ingest are optional features.

Shared resources: Archives that have entered into agreements with other Archives to share resources, perhaps to reduce cost. This requires various standards internal to the Archive (such as ingest-storage and access-storage interface standards) but does not alter the user community’s view of the Archive.

The remainder of this subsection gives a more detailed view of these categories of association.


6.1.1 INDEPENDENT ARCHIVES

An independent Archive is assumed to serve only a single Designated Community. The Archive and the Designated Community must agree on the design of DIPs and Finding Aids. An independent Archive may choose to design these structures based on formal or de-facto standards, which would allow cooperation with other Archives that implement the same standards. However, the design decisions to use these standards are not based on the possibility of inter-operation with other Archives, but rather on local requirements and cost savings.

The classification of an Archive as independent is not based on its size or distributed functionality. An independent Archive may occupy one site, or may be physically distributed over many sites. It may use many standards for a given internal element. However, if there is no interaction with other Archives, the Archive is independent.


6.1.2 COOPERATING ARCHIVES

Cooperating Archives are based on standards agreements among two or more Archives. The simplest form of cooperation between Archives is when one Archive acts as a Consumer of material from another Archive. In this case the consuming Archive must support the DIP format of the producing Archive as a SIP format. Cooperating Archives have related communities of interest, so they order and ingest data from other cooperating Archives and possibly have common data Producers. No common access, submission or dissemination standards are assumed. The only requirement for this architecture is that the cooperating groups support at least one common SIP and DIP format for inter-Archive requests. The control mechanism for this sort of inter-operation can be Event Based Order requests at each Archive.

Figures 6-1 and 6-2 illustrate the concept of cooperating Archives.

At a rudimentary level of Archive interaction, figure 6-1 represents a simple mutual information exchange agreement between Archives.

NOTE – In this and the following figures, the OAIS is represented as a multi-port device following the arrangement of figure 6-1. In each case, a two-Archive federation is shown for simplicity, although the concept can be extended indefinitely.

The essential requirement for this federation is a set of mutual Submission Agreements, Event Based Orders, and user interface standards to allow DIPs from one Archive to be ingested as SIPs by another. Therefore, it assumes that some pair-wise compatibility has between established between the Archives. This does not necessarily require common access, dissemination and submission methods for all participants, although that might encourage more exchange. This level of agreement would also be useful when the holdings of one Archive were consolidated/transferred into another Archive because of Management issues.

Figure 6-1: Cooperating Archives with Mutual Exchange Agreement

Figure 6-2 is an example of OAIS Archives that have standardized their submission and dissemination methods for the benefit of users. No special external element is needed for this. Its disadvantage is that there is no formal mechanism for exchange of Description Information so Consumers must have separate Search Sessions to locate AIPs of interest.

Figure 6-2: Cooperating Archives with Standard Ingest and Access Methods


6.1.3 FEDERATED ARCHIVES

Federated Archives are conceptually Consumer-oriented. In addition to the Local Community (i.e., the original Designated Community served by the Archive), there exists a Global Community (i.e., an extended Designated Community) which has interests in the holdings of several OAIS Archives and has influenced those Archives to provide access to their holdings via one or more common finding aids. However, the Local Consumers are likely to have access priority over the global Consumers.

At the federated level of association, external elements can be introduced to improve inter- operability. For example, figure 6-3 illustrates a functional architecture to solve the Access problem described in 6.1.2, using an entity external to the Federated OAISes. Here, two OAIS Archives that have similar Designated Communities have decided to federate to allow Consumers to locate Information Packages of interest from either OAIS with a single Search Session. The Common Catalog & Manager is the external (global) binding element that serves as a common access point for the information in both Archives. DIPs containing the finding aids from each OAIS are ingested into the Common Catalog as is shown by the dotted lines in figure 6-3. The Common Catalog may limit its activity to being a finding aid or it may include common dissemination of products from either or both OAISes as shown by the dashed lines in the figure.

NOTE – Extra access ports are added to the diagram to illustrate the potentially differing views of each consumer community.

Figure 6-3: An OAIS Federation Employing a Common Catalog

Federated Archives may be further classified into three levels of functionality.

– Central Site: Global access is accomplished by the export of a standard-format Associated Description to a global site. The global site independently manages a set of descriptors from many Archives and has finding aids to locate which Archive owns a collection of interest. The Consumer is given a combined view of the holdings of multiple sites, which is maintained centrally. To view details of the documents, the user must access the site that contains the actual document. This is made easier when sites and clients support a standard set of protocols.

– Distributed Finding Aid: Global access is accomplished by having a global node that can distribute a query to multiple local Archives. This means that the local Data Management entity must store an additional Associated Description in the global format or have a translator from the global queries to local queries. An option in this case is to establish a common DIP format to ease the load on Consumers who may be ordering products from many Archives.

– Distributed Access Aid: Global access is accomplished through addition of a standard ordering and dissemination mechanism, available through the global nodes to the functionality of the Distributed Finding Aids discussed above. This is a fully functional, federated system. Here, the global system may influence the Associated Description schema designs in each local Archive; it would be optimal to build new local Archives based on the global schemas and finding aids to ensure high degrees of inter-operability.

There are several major policy/technology issues that must be addressed when an OAIS joins a Federation or several independent OAISes decide to create a Federation. These issues include:

Unique AIP Names for each AIP in the Federation. An OAIS has the responsibility to provide each of its AIPs with a unique identity. When an OAIS joins a Federation, there is no assurance that some of its current OAIS AIP identifiers are not already used by other members of the Federation. An example of a general solution to this problem is to form the AIP identifiers in the Federation by assigning a Unique ID for each OAIS in the Federation and concatenating it to each AIP preserved by that OAIS. This OAIS name could be formatted according to a standard that gives the Customer or other Federation members the information needed to establish a connection to the OAIS that contains the AIP Interest. An example of a standard that accomplishes this is the ISO X.500 Directory Services Naming.

Duplicate AIPs in several different OAISes with different AIP names. This problem is caused by the fact that prior to Federation, some OAISes will have duplicated Content Information from AIPs in other OAISes to enable local user access. In this case a Global Consumer will see all the copies as separate, uniquely identified AIPs. Detailed examination of the PDI associated with the Content Information should allow the Consumer to locate the original, authoritative copy, but the search process could be very frustrating to the user. A practical way to handle this is to have a field in the Associated Description of all AIPs that identifies whether they are the original or a copy. This technique is not effective if, prior to federation, two or more Archives received the content information from the Producer to Archive. In this case the federated Archive would view these duplicate AIPs as unique, original AIPs.

The Preservation of Federation Access to AIPs when an OAIS terminates operations. Unfortunately, many Archives will close while their holdings still are of value to the Federation community. The federation should have an agreement for each member OAIS, which states which OAIS has the responsibility to take over the preservation of a closed OAIS’s holdings.

User Authentication and Access Management for global users. If an OAIS has a policy that restricts the access to some of its AIPs or charges for the dissemination of some Information Packages, there is a problem of how to identify and authenticate the user who is making requests through the central node. Each OAIS will have implemented an Authentication and Access Management system for its Local Users and the infrastructure for this function will have to be extended to include Global Users. Some examples of techniques used for this in current systems are:

• Default priorities where all members from the Global Node share a common set of access constraints and the Global Node handles all the authentication to verify the Consumer as a legitimate user of the Global node. The authentication at member OAIS is done assuming that all requests from the global node are from a single user.

• User Credential passing where the specific remote user can be authenticated by any of the Federation OAISes and the global node simply acts as an intermediary to carry the authentication dialog.

There are many factors influencing the decision of which of these techniques should be used by a specific Federation. The major criterion is the granularity of the Access Constraints in the Federation. If there is little private data and no charge for data dissemination, a policy that determines a user’s access constraints by the source he uses to discover and order AIPs is very reasonable. This involves little modification to the OAIS Authentication system, simply adding the Global Node as a Consumer. The Global Node will have to include mechanisms to identify Global Users to each of the Federated OAIS Authentication mechanisms.

If there are charges for disseminating archived information or significant private data that needs individual user authentication, the proxy techniques will not be sufficient, and User Credential passing techniques such as passwords and Certificates must be applied. The technologies to enable these supporting mechanisms are still evolving.


6.1.4 ARCHIVES WITH SHARED FUNCTIONAL AREAS

In an association involving Archives with shared functional areas, Management has entered into agreements with Archives to share or integrate functional areas. The motive for this may be to share expensive resources such as hierarchical file management system for Archival Storage, peripheral device for Ingest or dissemination of Information Packages or supercomputers for complicated transformations between SIPs, AIPs or DIPs. This association is fundamentally different from the previous examples, in that it is no longer possible to ignore the internal architecture of the Archive.

Figure 6-4 illustrates the sharing of a common storage function, consisting of an Archival Storage entity and a Data Management entity, between two Archives, OAIS 1 and OAIS 2. The Access and Ingest facilities can be at any of the previously described levels of inter- operability. In fact, each Archive can serve totally independent communities as implied in this figure. However, for the common storage element to succeed, standards are needed at the internal Ingest-storage and Access-storage interfaces.

Additional potential shared services include registries of Representation Information and name resolvers such as the DNS. In the former case a registry of Representation Information should also be an OAIS and the Representation Information it holds should be part of the Content Information it holds. The Representation Information it holds might, for example, be part of the Representation Network for the Content Information within an AIP in another OAIS. In such a case the OAIS holding that AIP may cache copies of the Representation Network held in the registry. Whether it does so or instead relies on the registry to maintain the Representation Network, the ultimate responsibility for the understandability of the Content Information remains with the OAIS which holds the AIP.

Figure 6-4: Archives with Shared Storage