Difference between revisions of "4.1 FUNCTIONAL MODEL"

From wiki.dpconline.org
Jump to navigation Jump to search
(Created page with "The OAIS of figure 2-1 is separated in figure 4-1 into six functional entities and related interfaces. Only major information flows are shown. The lines connecting entities id...")
 
Line 20: Line 20:




== 4.1.1 DETAILED DESCRIPTION OF FUNCTIONAL ENTITIES ==


In the following subsections, specific flows of information among the functional entities are identified in italics the first time they appear in the text. The detailed functional descriptions of the subsections are accompanied by diagrams (figures 4-2 through 4-7) that depict only the major data flows within and among the entities. Omitted for clarity are minor flows such as acknowledgment notices. Annex A contains a figure that combines figures 4-2 through
4-7 to demonstrate overall consistency. However, this is not to be taken as a recommended design or implementation, and actual implementations are not expected to have a one-to-one mapping to the functions shown, and may for example choose to combine functions or break out functionality differently.




== 4.1.1.1 Common Services ==
== 4.1.1.1 Common Services ==


Modern, distributed computing applications assume a number of supporting services such as inter-process communication, name services, temporary storage allocation, exception handling, security, backup and directory services. Much excellent work has already been done in the area of open system environment reference models. Examples of such services include:
'''Operating system services''' provide the core services needed to operate and administer the application platform, and provide an interface between application software and the platform. These services include the following:
– Kernel operations provide low-level services necessary to create and manage processes, execute programs, define and communicate signals, define and process system clock operations, manage files and directories, and control input-output processing to and from the external environment.
– Commands and utilities include mechanisms for operations at the operator level, such as comparing, printing, and displaying file contents; editing files; pattern searching; evaluating expressions; logging messages; moving files between directories; sorting data; executing command scripts; and accessing environment information.
– Real-time extension includes the application and operating system interfaces needed to support those application domains requiring deterministic execution, processing, and responsiveness. The extension defines the applications interface to basic system services for input/output, file system access, and process management.
– System management includes capabilities to define and manage user resource allocation and access (i.e., what resources are managed and the classes of access defined), configuration and performance management of devices, file systems, administrative processes (job accounting), queues, machine/platform profiles, authorization of resource usage, and system backup.
– Operating system security services specify the control of access to system data, functions, hardware, and software resources by users and user processes.
'''Network services''' provide the capabilities and mechanisms to support distributed applications requiring data access and applications interoperability in heterogeneous, networked environments. These services include the following:
– Data communication includes API and protocol specifications for reliable, transparent, end-to-end data transmission across communications networks.
– Transparent file access provides access to available files located anywhere in a heterogeneous network.
– Personal/micro computer support provides support for interoperability with systems based on other operating systems, particularly microcomputer operating systems, which may not be formally specified in a national or international standard.
– Remote Procedure Call services include specifications for extending the local procedure call to a distributed environment.
– Network security services include access, authentication, confidentiality, integrity, and non-repudiation controls and management of communications between senders and receivers of information in a network.
'''Security services''' provide capabilities and mechanisms to protect sensitive information and treatments in the information system. The appropriate level of protection is determined based upon the value of the information to the application end-users and the perception of threats to it. These services include the following:
– Identification/authentication service confirms the identities of requesters for use of information system resources. In addition, authentication can apply to providers of data. The authentication service may occur at the initiation of a session or during a session.
– Access control service prevents the unauthorized use of information system resources. This service also prevents the use of a resource in an unauthorized way. This service may be applied to various aspects of access to a resource (e.g., access to communications to the resource, the reading, writing, or deletion of an information/data resource, the execution of a processing resource) or to all accesses to a resource.
– Data integrity service ensures that data is not altered or destroyed in an unauthorized manner. This service applies to data in permanent data stores and to data in communications messages.
– Data confidentiality service ensures that data is not made available or disclosed to unauthorized individuals or computer processes. This service will be applied to devices that permit human interaction with the information system. In addition, this service will ensure that observation of usage patterns of communications resources will not be possible.
– Non-repudiation service ensures that entities engaging in an information exchange cannot deny being involved in it. This service may take one or both of two forms. First, the recipient of data is provided with proof of the origin of the data. This protects against any attempt by the sender to falsely deny sending the data or its contents. Second, the sender of data is provided with proof of delivery of data. This protects against any subsequent attempt by the recipient to falsely deny receiving the data or its contents.




== 4.1.1.2 Ingest ==
== 4.1.1.2 Ingest ==
The functions of the Ingest Functional Entity are illustrated in figure 4-2.
'''Figure 4-2: Functions of the Ingest Functional Entity'''
The '''Receive Submission''' function provides the appropriate storage capability or devices to receive a SIP from the Producer (or from Administration). Digital SIPs may be delivered via electronic transfer (e.g., FTP), loaded from media submitted to the Archive, or simply mounted (e.g., CD-ROM) on the Archive file system for access. Non-digital SIPs would likely be delivered by conventional shipping procedures. The Receive Submission function may represent a legal transfer of custody for the Content Information in the SIP, and may require that special access controls be placed on the contents. This function provides a confirmation of receipt of a SIP to the Producer, which may include a request to resubmit a SIP in the case of errors resulting from the SIP submission.
Evidence for Authenticity is provided by the Producer as part of the PDI in the submission, and this evidence is maintained, updated, and/or incremented by the Archive over time. The Producer may provide, or the Archive may itself define, as part of the Provenance Information, '''Information Property Descriptions''' of '''Information Properties''' which should be maintained over time, and indeed may provide Information Property Descriptions of Information Properties which do not need to be maintained over time. An Information Property is that part of the Content Information as described by the Information Property Description. An Information Property Description is a description of a part of the information content of a Content Information object that is highlighted for a particular purpose. The detailed expression, or value, of that part of the information content is conveyed by the appropriate parts of the Content Data Object and its Representation Information. For example, consider a simple digital book which when rendered appears as pages with margins, title, chapter headings, paragraphs, and text lines composed of words and punctuation. Information Property Descriptions for Information Properties that must be preserved could be expressed as
‘paragraph identification’ and ‘characters expressing words and punctuation’. The Information Properties would consist of all the book’s paragraph identifications, words, and punctuation as expressed by the Content Data Object and its Representation Information. This means that all formatting other than the recognition of paragraphs and readable text could be altered while still maintaining required preservation. The Archive may express an evaluation of the Authenticity of its holdings, based on community practice and recommendations (including best practices, guidelines, standards, and legal requirements). For example scientific Archives may have less stringent evaluation criteria than State Archives; however, the Consumer may make his/her own judgment of the Authenticity starting with the evidence obtained from PDI.
The '''Quality Assurance'''  function validates (QA results) the successful transfer of the SIP to the temporary storage area. For digital submissions, these mechanisms might include Cyclic Redundancy Checks (CRCs) or checksums associated with each data file, or the use of system log files to record and identify any file transfer or media read/write errors.
The '''Generate AIP''' function transforms one or more SIPs into one or more AIPs that conform to the Archive’s data formatting standards and documentation standards. This may involve file format conversions, gathering adequate Representation Information, data representation conversions or reorganization of the Content Information in the SIPs. The Generate AIP function may issue report requests to Data Management to obtain reports of information needed by the Generate AIP function to produce the Descriptive Information that completes the AIP. This function sends SIPs or AIPs for audit to the Audit Submission function in Administration, and receives back an audit report. As a result of the audit report for example, it may be necessary to gather further Representation Information to ensure that the Content Information is understandable and usable by the Designated Community
The '''Generate Descriptive Information''' function extracts Descriptive Information from the AIPs and collects Descriptive Information from other sources to provide to Coordinate Updates, and ultimately Data Management. This includes metadata to support searching and retrieving AIPs (e.g., who, what, when, where, why), and could also include special browse products (thumbnails, images) to be used by Finding Aids.
The '''Coordinate Updates''' function is responsible for transferring the AIPs to Archival Storage and the Descriptive Information to Data Management. Transfer of the AIP includes a storage request and may represent an electronic, physical, or a virtual (i.e., data stays in place) transfer. After the transfer is completed and verified, Archival Storage returns a storage confirmation indicating (or verifying) the storage identification information for the AIP. The Coordinate Updates function also incorporates the storage identification information into the Descriptive Information for the AIP and transfers it to the Data Management entity along with a database update request. In return, Data Management provides a database update response indicating the status of the update. Data Management updates may take place without a corresponding Archival Storage transfer when the SIP contains Descriptive Information for an AIP already in Archival Storage.




Line 93: Line 33:
== 4.1.1.3 Archival Storage ==
== 4.1.1.3 Archival Storage ==


The functions of the Archival Storage Functional Entity are illustrated in figure 4-3. The term ‘media’ is used to designate one or more mechanisms, local or remote, for storing digitally encoded information.
'''Figure 4-3: Functions of the Archival Storage Functional Entity'''
The '''Receive Data''' function receives a storage request and an AIP from Ingest and moves the AIP to permanent storage within the Archive. The transfer request may need to indicate the anticipated frequency of utilization of the Data Objects making up the AIP in order to allow the appropriate storage devices or media to be selected for storing the AIP. This function will select the media type, prepare the devices or volumes, and perform the physical transfer to the Archival Storage volumes. Upon completion of the transfer, this function sends a storage confirmation message to Ingest, including the storage identification of the AIPs.
The '''Manage Storage Hierarchy''' function positions, via commands, the contents of the AIPs on the appropriate media based on storage management policies, operational statistics, or directions from Ingest via the storage request. It will also conform to any special levels of service required for the AIP, or any special security measures that are required, and ensures the appropriate level of protection for the AIP. These include on-line, off-line or near-line storage, required throughput rate, maximum allowed bit error rate, or special handling or backup procedures. It monitors error logs to ensure AIPs are not corrupted during transfers. This function also provides operational statistics to Administration summarizing the inventory of media on-hand, available storage capacity in the various tiers of the storage hierarchy, and usage statistics.
The '''Replace Media''' function provides the capability to reproduce the AIPs over time. Within the Replace Media function the Content Information and Preservation Description Information (PDI) must not be altered. However, the data constituting the Packaging Information may be changed as long as it continues to perform the same function and there is
a straightforward implementation that does not cause information loss. The migration strategy must select a storage medium, taking into consideration the expected and actual rates of errors encountered in various media types, their performance, and their costs of ownership. If media-dependent attributes (e.g., tape block sizes, CD-ROM volume information) have been included as part of the Content Information, a way must be found to preserve this information when migrating to higher capacity media with different storage architectures. Anticipating the terminology of 5.1.3, this function may perform
‘Refreshment’, ‘Replication’, and ‘Repackaging’ that is straightforward. An example of such ‘Repackaging’ is migration to new media under a new operating system and file system, where the Content Information and PDI are independent of the file systems. However, complex ‘Repackaging’ and all ‘Transformation’ are performed under Administration supervision by the Archival Information Update function to ensure information preservation. (Refer to 5.1.3 for a detailed description of migration issues.)
The '''Error Checking''' function provides statistically acceptable assurance that no components of the AIP are corrupted in Archival Storage or during any internal Archival Storage data transfer. This function requires that all hardware and software within the Archive provide notification of potential errors and that these errors are routed to standard error logs that are checked by the Archival Storage staff. The PDI Fixity Information provides some assurance that the Content Information has not been altered as the AIP is moved and accessed. Similar information is needed to protect the PDI itself. A standard mechanism for tracking and verifying the validity of all Data Objects within the Archive may also be used. For example, CRCs could be maintained for every individual data file. A higher level of service, such as Reed-Solomon coding to support combined error detection and correction, could also be provided. The storage facility procedures should provide for random verification of the integrity of Data Objects using CRCs or some other error checking mechanism.
The '''Disaster Recovery''' function provides a mechanism for duplicating the digital contents of the Archive collection and, for example, storing the duplicate in a physically separate facility. This function is normally accomplished by copying the Archive contents to some form of removable storage media (e.g., digital linear tape, CD-ROM), but may also be performed via hardware transport or network data transfers. The details of disaster recovery policies are specified by Administration.
The '''Provide Data''' function provides copies of stored AIPs to Access. This function receives an AIP request that identifies the requested AIP(s) and provides them on the requested media type or transfers them to a temporary storage area. This function also sends a notice of data transfer to Access upon completion of an order.




== 4.1.1.4 Data Management ==
== 4.1.1.4 Data Management ==
The functions of the Data Management Functional Entity are illustrated in figure 4-4.
'''Figure 4-4: Functions of the Data Management Functional Entity'''
The '''Administer Database''' function is responsible for maintaining the integrity of the data management database, which provides a storage mechanism, which can be queried in some way, for storing both Descriptive Information and system information. Descriptive Information identifies and describes the Archive holdings, and system information is used to support Archive operations. The Administer Database function is responsible for creating any schema or table definitions required to support Data Management functions; for providing the capability to create, maintain and access customized user views of the contents of this storage; and for providing internal validation (e.g., referential integrity) of the contents of the database. The Administer Database function is carried out in accordance with policies received from Administration.
The Perform Queries function receives a query request from Access and executes the query to generate a query response that is transmitted to the requester.
The '''Generate Report''' function receives a report request from Ingest, Access or Administration and executes any queries or other processes necessary to generate the report that it supplies to the requester. Typical reports might include summaries of Archive holdings by category, or usage statistics for accesses to Archive holdings. It may also receive a report request from Access and provides descriptive information for a specific AIP.
The '''Receive Database Updates''' function adds, modifies or deletes information in the Data Management persistent storage. The main sources of updates are Ingest, which provides Descriptive Information for the new AIPs, and Administration, which provides system updates and review updates. Ingest transactions consist of Descriptive Information which identifies new AIPs stored in the Archive. System updates include all system-related information (operational statistics, Consumer information, and request status). Review updates are generated by periodic reviewing and updating of information values (e.g., contact names, addresses, access control and rights policies). The Receive Database Updates function provides regular reports to Administration summarizing the status of updates to the database, and also sends a database update response to Ingest.




== 4.1.1.5 Administration ==
== 4.1.1.5 Administration ==
The functions of the Administration Functional Entity are illustrated in figure 4-5. As stated above (see page 4-3) some of the activities detailed in this section will not apply to all implementations; for instance in the customer service function, the billing activity shows where this would occur if required.
'''Figure 4-5: Functions of the Administration Functional Entity'''
The '''Negotiate Submission Agreement''' function solicits desirable archival information for the OAIS and negotiates Submission Agreements with Producers. This function also negotiates a data submission schedule with the Producer. It maintains a calendar of expected Data Submission Sessions that will be needed to transfer one or more SIPs to the OAIS and the resource requirements to support their ingestion. This function receives AIP/SIP templates and customization advice from Preservation Planning and sends SIP designs, any customized AIP designs and any example SIPs to the Audit Submission function as part of the submission approval process. It also sends SIP designs and any customized AIP designs to the Establish Standards and Policies function for subsequent use by the Ingest function. The data submission formats and procedures must be clearly documented in the Archive’s data submission policies, and the deliverables must be identified by the Producer in the Submission Agreement. The Producer-Archive Interface Methodology Abstract Standard (ISO 20652:2006) may be applicable here, as may be parts of ISO 15489-1:2001 and ISO/TR
15489-2:2001, and other ISO standards.
The '''Manage System Configuration''' function provides system engineering for the Archive system to monitor continuously the functionality of the entire Archive system and systematically control changes to the configuration. This function maintains integrity and traceability of the configuration during all phases of the system life cycle. It also audits system operations, system performance, and system usage. It sends report requests for system information to Data Management and receives reports; it receives operational statistics from Archival Storage. It summarizes those reports and periodically provides OAIS performance information and Archive holding inventory reports to Preservation Planning. It receives migration packages from Preservation Planning, based on which it sends change requests, procedures and tools to the Archival Information Update function. It receives system evolution policies from the Establish Standards and Policies function. Based on these inputs it develops and implements plans for system evolution.
The '''Archival Information Update''' function provides a mechanism for updating the contents of the Archive. It receives change requests, procedures and tools from Manage System Configuration. It provides updates by sending a dissemination request to Access, updating the contents of the resulting DIPs and resubmitting them as SIPs to Ingest.
The '''Physical Access Control''' function provides mechanisms to restrict or allow physical access (doors, locks, guards) to elements of the Archive, as determined by Archive policies.
The '''Establish Standards and Policies''' function is responsible for establishing and maintaining the Archive system standards and policies. It receives budget information and policies such as the OAIS charter, scope, resource utilization guidelines, and pricing policies from Management. It provides Management with periodic reports. It receives recommendations for Archive system enhancement, proposals for new Archive data standards, and periodic risk analysis reports from Preservation Planning. It will have to face risks from unforeseen events (unplanned down time due to network outage, software bugs, hardware failure, human error, disk crash, etc.) and make the appropriate decisions to minimize the risk of not fulfilling the Archive’s commitments. It also receives performance information and Archive holding inventories from Manage System Configuration. Based on these inputs and analyses, Archive standards and policies are established and sent to other Administration functions and the other Functional Entities for implementation. The standards include format standards, documentation standards and the procedures to be followed during the Ingest process. In response to the recommendations from Preservation Planning on AIP updates it provides approved standards and migration goals to Preservation Planning (Preservation Planning will respond with migration packages to the Manage System Configuration function). The Establish Standards and Policies function will also develop storage management policies (for the Archival Storage hierarchy), including migration policies to assure that Archive storage formats do not become obsolete, and database administration policies. It will develop disaster recovery policies. It will also determine security policies for the contents of the Archive, including those affecting Physical Access Control, such as DRM, and the application of error control techniques throughout the Archive.
The '''Audit Submission''' function will verify that submissions (SIP or AIP) meet the specifications of the Submission Agreement. In the case of the SIP and in the case of the AIP it verifies the understandability by the Designated Community. This function receives AIP/SIP reviews from Preservation Planning and may also involve an outside committee (e.g., science and technical review); these reviews report on whether the AIP/SIP templates have been properly applied in Ingest. The audit process must verify that the quality of the data meets the requirements of the Archive and the review committee. It must verify that there is adequate Representation Information and PDI to ensure that the Content Information is understandable and independently usable to the Designated Community. The formality of the review will vary depending on internal Archive policies. The Audit process may determine that some portions of the SIP are not appropriate for inclusion in the Archive and must be resubmitted or excluded. An audit report is provided to Ingest. After the audit process is completed, any liens are reported to the Producer, who will then resubmit the SIP to Ingest or appeal the decision to Administration. In the case of producing a new version of an AIP this function checks that the migration goals have been met. This could include checking usability and checking evidence for Authenticity such as ensuring maintenance of Transformational Information Properties (see 5.2). After the audit is completed, a final ingest report is prepared and provided to the Producer and to Negotiate Submission Agreement. Audit methods potentially include sampling, periodic review, and peer review.
The '''Activate Requests''' function maintains a record of event-driven requests and periodically compares it to the contents of the Archive to determine if all needed data is available. If needed data is available, this function generates a dissemination request that is sent to Access. This function can also generate orders on a periodic basis where the length of the period is defined by the Consumers or on the occurrence of an event (e.g., a database update).
The '''Customer Service''' function will create, maintain and delete Consumer accounts. It will collect billing information from Access and will send bills and collect payment from Consumers for the utilization of Archive system resources. It will respond to general information requests. This function will also collect and respond to feedback on Access services and products. Customer Service will summarize these comments and make them available.


== 4.1.1.6 Preservation Planning ==
== 4.1.1.6 Preservation Planning ==
The functions of the Preservation Planning Functional Entity are illustrated in figure 4-6.
'''Figure 4-6: Functions of the Preservation Planning Functional Entity'''
The '''Monitor Designated Community''' function interacts with Archive Consumers and Producers to track changes in their service requirements and available product technologies. Such requirements might include data formats, media choices, preferences for software packages, new computing platforms, and mechanisms for communicating with the Archive. This function may be accomplished via surveys, via a periodic formal review process, via community workshops where feedback is solicited or by individual interactions. It provides reports, requirements alerts and emerging standards to the Develop Preservation Strategies and Standards function. It sends preservation requirements to Develop Packaging Designs.
The '''Monitor Technology''' function is responsible for tracking emerging digital technologies, information standards and computing platforms (i.e., hardware and software) to identify technologies which could cause obsolescence in the Archive’s computing environment and prevent access to some of the Archive’s current holdings. This function may contain a prototyping capability for better evaluation of emerging technologies and receive prototype requests from Develop Preservation Strategies and Standards and from Develop Package
Designs and Migration Plans. This function sends reports, external data standards, prototype results and technology alerts to Develop Preservation Strategies and Standards. It also sends prototype results to Develop Package Designs and migration plans.
The '''Develop Preservation Strategies and Standards''' function is responsible for developing and recommending strategies and standards, and for assessing risks, to enable the Archive to make informed tradeoffs as it establishes standards, sets policies, and manages its system infrastructure. Risk management is a suitable methodology to provide balance between needs and means, and between immediate activity imperatives and Long Term objectives of the preservation mission. Risk management can also provide useful metrics to quantify elements that are usually difficult to estimate in the decision-making process. This function provides periodic risk analysis reports to Administration addressing expected risks and their possible mitigation based on current, and on proposed updates, to operating policies, procedures and standards. This function receives reports, requirements alerts and emerging standards from the Monitor Designated Communities and Monitor Technology functions, and it receives operating policies, procedures and standards, performance information, inventory reports and summarized consumer comments from Administration. From the information this function identifies those changes that would require migration of some current Archive holdings or new submissions, for example updating AIPs with additional or revised Representation Information. This function sends recommendations on system evolution and on AIP updates to Administration This function also receives external data standards from Monitor Technology and produces profiles of those standards that are sent to Administration as proposals on their potential usage. This function also receives issues from Develop Packaging Designs and migration plans in the case of unanticipated submission requirements, and responds with advice to handle the new requirements.
The '''Develop Packaging Designs and Migration Plans''' function develops new Information Package designs and detailed migration plans and prototypes, to implement Administration policies and directives. Migration of the Content Information could involve changes to the Content Data Object and/or the Representation Information. This activity also provides advice on the application of these Information Package designs and Migration plans to specific Archive holdings and submissions. This function receives Archive approved standards and migration goals from Administration. The standards include format standards, metadata standards and documentation standards. It applies these standards to preservation requirements and provides AIP and SIP template designs to Administration. This function also provides customization advice and AIP/SIP review to Administration on the application of those designs. If this function encounters submissions that are not covered by existing standards and procedures, it can send issues to Develop Preservation Strategies and Standards and receive advice, including new standards, to assist in meeting the new submission requirements.
The preservation requirements and the migration goals received by this function tend to involve transformations of the AIP, including transformations of the Content Information to avoid loss of access due to technology obsolescence (see section 5). The response to the migration goals may involve the development of new AIP designs, prototype software, test plans, community review plans and implementation plans for phasing in the new AIPs. This process may call on expertise or resources from other functions within Preservation Planning, such as prototype development from Monitor Technology. This effort also will require consultation from the other functional areas and from the Designated Community. Once the migration plan, associated AIP designs, and software have been tested and approved by Administration, this function will send the entire migration package to Administration. These proposals for the migration plan are received and approval granted (or denied) by the ‘Establish Standards and Policies’ function of ‘Administration’. The
‘Preservation Planning’ entity develops, validates and supplies the migration packages on the basis of this approval; Administration schedules and performs the migration plans.




== 4.1.1.7 Access ==
== 4.1.1.7 Access ==
The functions of the Access Functional Entity are illustrated in figure 4-7.
'''Figure 4-7: Functions of the Access Functional Entity'''
The '''Coordinate Access Activities''' function provides one or more interfaces to the information holdings of the Archive. This interface will normally be via computer network or dial-up link to an on-line service, but might also be implemented in the form of a walk-in facility, printed catalog ordering service, or fax-back type service. Three categories of Consumer requests are distinguished: query requests, which are executed in Data Management and return immediate query responses for presentation to the user; report requests, which may require a number of queries and produce formatted reports for delivery to the Consumer; and orders, which may access either or both Data Management and Archival Storage to prepare a formal Dissemination Information Package (DIP) for on- or off-line delivery. An order may be an Adhoc Order that is executed only once, or an Event Based Order that will be maintained by the Activate Requests function in Administration, and initiated by a dissemination request that may result in periodic deliveries of requested items. The Archival Information Update function in Administration also submits dissemination requests to obtain DIPs needed to perform its update functions. Other special request types are allowed, but are not detailed. This function will determine if resources are available to perform a request, assure that the user is authorized to access and receive the requested items, and notify the Consumer that a request has been accepted or rejected (possibly with an estimate of request cost and an option to cancel the request). It will then transfer the request to Data Management or to the Generate DIP function for execution. This function also provides assistance to OAIS Consumers including providing status of orders and other Consumer support activities in response to an assistance request.
The '''Generate DIP''' function accepts a dissemination request, retrieves the AIP from Archival Storage, and moves a copy of the data to a temporary storage area for further processing. This function also transmits a report request to Data Management to obtain Descriptive Information needed for the DIP. If special processing is required, the Generate DIP function accesses Data Objects in temporary storage and applies the requested processes. The types of operations which may be carried out include statistical functions, sub-sampling in temporal or spatial dimensions, conversions between different data types or output formats, and other specialized processing (e.g., image processing). Inserting DRM information and filtering the personal data to ensure consistency with the user rights also come under this type of operation. This function places the completed DIP response in the temporary storage area and notifies the Coordinate Access Activities function that the DIP is ready for delivery. It is worth noting that in some implementations the AIP content or the DIPs could be kept in temporary storage for ready availability.
The '''Deliver Response''' function handles both on-line and off-line deliveries of responses (DIPs, query responses, reports and assistance) to Consumers. For on-line delivery, it accepts a response from Coordinate Access Activities and prepares it for on-line distribution in real time via communication links. It identifies the intended recipient, determines the transmission procedure requested, places the response in the temporary storage area to be transmitted, and supports the on-line transmission of the response. For off-line delivery it retrieves the response from the Coordinate Access Activities function, prepares packing lists and other shipping records, and then ships the response. When the response has been shipped, a notice of shipped order is returned to the Coordinate Access Activities function and billing information is submitted to Administration.




== 4.1.2 DATA FLOW DIAGRAMS ==
== 4.1.2 DATA FLOW DIAGRAMS ==
The flow of data items among the OAIS functional entities is diagrammed in this subsection. Figure 4-8 shows the more significant data flows. To avoid complicating this figure, the Administration data flows, which are generally background activities, are isolated to an Administration context diagram, figure 4-9. Data flows associated with Common Services are implicit in the illustrated functions, and are therefore not shown.
'''Figure 4-8: OAIS Data Flow Diagram'''
'''Figure 4-9: Administration Context Diagram'''
A significant data flow that is not obvious is that which involves the update of AIPs in order to ensure they are adequate for preservation of the Content Information.
For example consider the case of needing to add Representation Information because of changes in the Knowledge Base of the Designated Community. Although the Data Object that is being preserved is not being changed, the need to change the Representation Information means the Content Information is being changed. This is in general terms a Migration of the AIP.
– Monitoring of the Designated Community and general environment (reports, requirements alerts, emerging standards, external data standards, prototype results and technology alerts) shows the need to update Representation Information for a particular AIP, which could itself be a collection of objects.
– Preservation Planning provides recommendations, proposals and risk analysis reports to Administration which evaluates the recommendations and decides what options to look at in more detail, and sends that decision to Preservation Planning as approved standards, preservation requirements and migration goals.
– Preservation Planning then produces detailed plans based on the instructions from Administration and passes these detailed plans back (AIP templates, migration packages and customization advice) to Administration.
– Administration then implements these plans (via Manage System Configuration). In general this will require an update (logically or physically) of one or more AIPs. The next steps may be done logically; i.e., the AIP does not necessarily have to be moved in any way.
– Instructions are then sent to Access (dissemination request) to send the original AIP out as a DIP which is then received by Ingest, together with the updated Representation Information.
– On Ingest a new AIP version is created (in Generate AIP) containing the updated
Representation Information.
'''--Please retain original text above for reference. Comment or propose amendments or additions below this line--'''

Revision as of 12:51, 13 August 2015

The OAIS of figure 2-1 is separated in figure 4-1 into six functional entities and related interfaces. Only major information flows are shown. The lines connecting entities identify communication paths over which information flows in both directions. The lines to Administration and Preservation Planning are dashed only to reduce diagram clutter.

Figure 4-1: OAIS Functional Entities

The role provided by each of the entities in figure 4-1 is described briefly as follows:

The Ingest Functional Entity (labeled ‘Ingest’ in the figures in this section) provides the services and functions to accept Submission Information Packages (SIPs) from Producers (or from internal elements under Administration control) and prepare the contents for storage and management within the Archive. Ingest functions include receiving SIPs, performing quality assurance on SIPs, generating an Archival Information Package (AIP) which complies with the Archive’s data formatting and documentation standards, extracting Descriptive Information from the AIPs for inclusion in the Archive database, and coordinating updates to Archival Storage and Data Management.

The Archival Storage Functional Entity (labeled ‘Archival Storage’ in the figures in this section) provides the services and functions for the storage, maintenance and retrieval of AIPs. Archival Storage functions include receiving AIPs from Ingest and adding them to permanent storage, managing the storage hierarchy, refreshing the media on which Archive holdings are stored, performing routine and special error checking, providing disaster recovery capabilities, and providing AIPs to Access to fulfill orders.

The Data Management Functional Entity (labeled ‘Data Management’ in the figures in this section) provides the services and functions for populating, maintaining, and accessing both Descriptive Information which identifies and documents Archive holdings and administrative data used to manage the Archive. Data Management functions include administering the Archive database functions (maintaining schema and view definitions, and referential integrity), performing database updates (loading new descriptive information or Archive administrative data), performing queries on the data management data to generate query responses, and producing reports from these query responses.

The Administration Functional Entity (labeled ‘Administration’ in the figures in this section) provides the services and functions for the overall operation of the Archive system. Administration functions include soliciting and negotiating submission agreements with Producers, auditing submissions to ensure that they meet Archive standards, and maintaining configuration management of system hardware and software. It also provides system engineering functions to monitor and improve Archive operations, and to inventory, report on, and migrate/update the contents of the Archive. It is also responsible for establishing and maintaining Archive standards and policies, providing customer support, and activating stored requests.

The Preservation Planning Functional Entity (labeled ‘Preservation Planning’ in the figures in this section) provides the services and functions for monitoring the environment of the OAIS, providing recommendations and preservation plans to ensure that the information stored in the OAIS remains accessible to, and understandable by, the Designated Community over the Long Term, even if the original computing environment becomes obsolete. Preservation Planning functions include evaluating the contents of the Archive and periodically recommending archival information updates, recommending the migration of current Archive holdings, developing recommendations for Archive standards and policies, providing periodic risk analysis reports, and monitoring changes in the technology environment and in the Designated Community’s service requirements and Knowledge Base. Preservation Planning also designs Information Package templates and provides design assistance and review to specialize these templates into SIPs and AIPs for specific submissions. Preservation Planning also develops detailed Migration plans, software prototypes and test plans to enable implementation of Administration migration goals.

The Access Functional Entity (labeled ‘Access’ in the figures in this section) provides the services and functions that support Consumers in determining the existence, description, location and availability of information stored in the OAIS, and allowing Consumers to request and receive information products. Access functions include communicating with Consumers to receive requests, applying controls to limit access to specially protected information, coordinating the execution of requests to successful completion, generating responses (Dissemination Information Packages, query responses, reports) and delivering the responses to Consumers.

In addition to the entities described above, there are various Common Services assumed to be available. These services are considered to constitute another functional entity in this model. This entity is so pervasive that, for clarity, it is not shown in figure 4-1.



4.1.1.1 Common Services

4.1.1.2 Ingest

4.1.1.3 Archival Storage

4.1.1.4 Data Management

4.1.1.5 Administration

4.1.1.6 Preservation Planning

4.1.1.7 Access

4.1.2 DATA FLOW DIAGRAMS