Pre-ingest: Difference between revisions

From wiki.dpconline.org
Jump to navigation Jump to search
(Should there be a Pre-Ingest stage in the OAIS model?)
 
(Added text from the AMIA/DLF HackDay OAIS Edit-A-Thon)
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
__NOTOC__
{{:OAISheader}}
The OAIS model supposes that the ingest procedure starts with SIPs that. after some handling become AIPs and can be ingested. What I’ve seen on many conferences and also experienced in our own organisation (KB-NL) is the need for an extra stage in the ingest procedure: pre –ingest. In this pre-ingest stage the received material will be checked on various aspects, that are fundamental to make a decision whether the material should be accepted to enter the repository in the first place, like
The OAIS model supposes that the ingest procedure starts with SIPs that. after some handling become AIPs and can be ingested. What I’ve seen on many conferences and also experienced in our own organisation (KB-NL) is the need for an extra stage in the ingest procedure: pre –ingest. In this pre-ingest stage the received material will be checked on various aspects, that are fundamental to make a decision whether the material should be accepted to enter the repository in the first place, like
*Does it fit the collection building criteria (you don’t want to start the ingest procedure for material that is not meeting these requirements)
*Does it fit the collection building criteria (you don’t want to start the ingest procedure for material that is not meeting these requirements)
Line 4: Line 7:
*An action need to be done to unpack a batch file with hundreds of objects and create a set of SIPs to be handled one by one, perhaps by creating SIPs by merging various parts of the batch (object, metadata)
*An action need to be done to unpack a batch file with hundreds of objects and create a set of SIPs to be handled one by one, perhaps by creating SIPs by merging various parts of the batch (object, metadata)
Another reason to add a specific pre-ingest function to OAIS is that organisations design working processes in which they deliberately want to distinguish the “raw material” received from the producer from the material that is being processed to become a SIP – with a name that they can use in their organisation, like Pre-Submission (Information)  Package: PSP
Another reason to add a specific pre-ingest function to OAIS is that organisations design working processes in which they deliberately want to distinguish the “raw material” received from the producer from the material that is being processed to become a SIP – with a name that they can use in their organisation, like Pre-Submission (Information)  Package: PSP
[[User:BSierman|Barbara Sierman]] ([[User talk:BSierman|talk]]) 14:09, 30 September 2015 (BST)
[[User:Eld|Eld Zierau]] :
It would also be good to take the OO-IO model (for Ingest as an inner OAIS) into consideration for this topic - discussed under blog http://wiki.dpconline.org/index.php?title=Outer_OAIS_-_Inner_OIAS_for_distribution_aspects
[[User:Shirapeltzman|Shira Peltzman]] : Below is some text on this subject generated during the AMIA/DLF HackDay OAIS Edit-A-Thon
When discussing the “Receive Submission” function of the Ingest Functional Entity (4.1.1.2) the standard elides the often onerous, time-consuming appraisal process. Although the audit process is given a cursory acknowledgment in the Ingest Functional Entity diagram (4.1.1.2), there is not very much text devoted to describing what happens during this process, or what this process should encompass.
This is problematic, because the reality is that many archives--particularly archives with a substantial amount of moving image and/or complex born-digital collections--frequently receive material from donors that contain a wide variety of file formats, project versions, and assorted production material. When this material arrives it is not possible to ingest or accession it without first reviewing, assessing, and preparing it. In other words, many attendees summed this up by noting that, “you don’t get SIP from donor; [you] end up doing a lot of the work to create the SIP”. This phase wherein the SIP is created is not only the one that archives often spend substantial time completing, but it’s also a critical part of the archiving process.
Even more crucially, however, there are no universally acknowledged tools, workflows, or resources devoted to addressing this phase.
This is due in part to the fact that there is not officially a distinct phase in the OAIS reference model that addresses this, and it would be helpful is the revised standard included a section which acknowledges that pre-ingest is a major part of the archive’s responsibilities.
There is another CCSDS standard (Producer-Archive Interface Specification) that addresses the processes around transferring a SIP in more detail. But given how critical and time-consuming (not to mention pervasive) the appraisal process is, it is important that the OAIS standard should address this step more fully. There are three ways in which it might do this: 1) by including language in section 4.1.1.2 that acknowledges this step and some of the processes involved in it (like making further judgments about the objects you receive, conducting additional research, etc.), 2) by illustrating the points of connection between these two standards, or 3) by adding an additional entity to the diagram acknowledging the fact that may be an additional Pre-Submission Info Packet (Pre-SIP? PSIP?) process through which an archives needs to appraise,organize, or make further judgments about the objects they receive before they can even call it a SIP.
[[Category:Blog]]
<!-- Ignore this stuff: -->
[[Category:Blog]]
{{:OAISfooter}}

Latest revision as of 22:31, 4 April 2016

OAIS Community Logo small.png

Community Forum | OAIS Community | OAIS Structure | OAIS Blog Posts | Active Topics and News

The OAIS model supposes that the ingest procedure starts with SIPs that. after some handling become AIPs and can be ingested. What I’ve seen on many conferences and also experienced in our own organisation (KB-NL) is the need for an extra stage in the ingest procedure: pre –ingest. In this pre-ingest stage the received material will be checked on various aspects, that are fundamental to make a decision whether the material should be accepted to enter the repository in the first place, like

  • Does it fit the collection building criteria (you don’t want to start the ingest procedure for material that is not meeting these requirements)
  • Is the purpose of the set of objects to be ingested (or is it wrongly delivered)
  • An action need to be done to unpack a batch file with hundreds of objects and create a set of SIPs to be handled one by one, perhaps by creating SIPs by merging various parts of the batch (object, metadata)

Another reason to add a specific pre-ingest function to OAIS is that organisations design working processes in which they deliberately want to distinguish the “raw material” received from the producer from the material that is being processed to become a SIP – with a name that they can use in their organisation, like Pre-Submission (Information) Package: PSP

Barbara Sierman (talk) 14:09, 30 September 2015 (BST)


Eld Zierau : It would also be good to take the OO-IO model (for Ingest as an inner OAIS) into consideration for this topic - discussed under blog http://wiki.dpconline.org/index.php?title=Outer_OAIS_-_Inner_OIAS_for_distribution_aspects

Shira Peltzman : Below is some text on this subject generated during the AMIA/DLF HackDay OAIS Edit-A-Thon

When discussing the “Receive Submission” function of the Ingest Functional Entity (4.1.1.2) the standard elides the often onerous, time-consuming appraisal process. Although the audit process is given a cursory acknowledgment in the Ingest Functional Entity diagram (4.1.1.2), there is not very much text devoted to describing what happens during this process, or what this process should encompass.

This is problematic, because the reality is that many archives--particularly archives with a substantial amount of moving image and/or complex born-digital collections--frequently receive material from donors that contain a wide variety of file formats, project versions, and assorted production material. When this material arrives it is not possible to ingest or accession it without first reviewing, assessing, and preparing it. In other words, many attendees summed this up by noting that, “you don’t get SIP from donor; [you] end up doing a lot of the work to create the SIP”. This phase wherein the SIP is created is not only the one that archives often spend substantial time completing, but it’s also a critical part of the archiving process.

Even more crucially, however, there are no universally acknowledged tools, workflows, or resources devoted to addressing this phase. This is due in part to the fact that there is not officially a distinct phase in the OAIS reference model that addresses this, and it would be helpful is the revised standard included a section which acknowledges that pre-ingest is a major part of the archive’s responsibilities.

There is another CCSDS standard (Producer-Archive Interface Specification) that addresses the processes around transferring a SIP in more detail. But given how critical and time-consuming (not to mention pervasive) the appraisal process is, it is important that the OAIS standard should address this step more fully. There are three ways in which it might do this: 1) by including language in section 4.1.1.2 that acknowledges this step and some of the processes involved in it (like making further judgments about the objects you receive, conducting additional research, etc.), 2) by illustrating the points of connection between these two standards, or 3) by adding an additional entity to the diagram acknowledging the fact that may be an additional Pre-Submission Info Packet (Pre-SIP? PSIP?) process through which an archives needs to appraise,organize, or make further judgments about the objects they receive before they can even call it a SIP.


OAIS Community Logo small.png

Community Forum | OAIS Community | OAIS Structure | OAIS Blog Posts | Active Topics and News

These wiki pages are licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License. Attribute as "Community forum for digital preservation and curation standards http://wiki.dpconline.org/". The content on this wiki represents the opinions of the author and not the Digital Preservation Coalition. This wiki is not associated with ISO, the OAIS Standard or the CCSDS.