Recently on NARAtions we announced that the Ancestry partnership agreement was up for renewal and available for public comment until August 21, 2015. We received almost 70 responses (thank you!), from which a number of themes emerged. Digitization Division director, Markus Most, previously addressed a few of these concerns as an update to that post. These concerns are again listed below:
How will records digitized by a partner be available to the public? ?
Records digitized and made available by partners are available online for free in all NARA research rooms across the country. You can use our public access PCs or NARA’s wifi to reach the sites for free.
?Will NARA receive digital copies of the material digitized by the partners? ?
NARA receives a set of images and metadata from the partner when digitization of materials is complete.
?Will NARA put the digitized material online? ?
NARA makes records digitized by Ancestry available in our online Catalog once the five year embargo period has elapsed. Records in the National Archives Catalog are available for free to the public.
?Why does NARA partner with outside organizations? ?
To provide increased access to our records, we have partnered with both for-profit and non-profit organizations on digitization. Although they may be interested in genealogy records, the specific records these organizations are interested in can vary. Sometimes only one type of partner may be interested in a record series. We digitize in order to get our records online and expand access to them. We cannot do this by ourselves and so we are working with partners and looking into other avenues (see our Digitization Strategy for additional approaches) to make access happen. NARA has established principles for working with partners and you can read them at our NARA Principles for Partnerships page. (See our Strategic Plan for more information about our Strategic Goals.)
Two additional themes emerged from researcher feedback that we would like to respond to:
The embargo period is over for some partner images. Where are those images in NARA’s online catalog?
In 2013, NARA posted the first 250,000 partner images that could be released from the 2007 digitization year. Since then, we have continued to add images and additional capacity to handle the tremendous growth of the National Archives Catalog. There are currently over 5.25 million images in the Catalog. A significant proportion of these images come from our partner projects with new images added to the catalog on an ongoing basis.
Additionally, the partners have undertaken many digitization projects outside of the digitization partnerships, particularly before the partnerships were established. Partners oftentimes purchased microfilm publications, digitized them, and made them available on their websites. Because these images were produced outside of an agreement, NARA does not receive a copy. For example, the Revolutionary War pensions were digitized prior to a partnership agreement and are therefore not available through our catalog.
What quality control measures are in place to address indexing quality, image quality, and scanning accuracy?
Our quality control procedures currently are:
QC of imaging is the responsibility of the partner, following standards reported to, and approved by, NARA. The precise standards are proprietary information.
QC of metadata is the responsibility of the partner, following standards reported to, and approved by, NARA. The precise standards are proprietary information.
QC of content is the responsibility of NARA – Specifically, NARA does a page-by-page review against a five percent sample of the original records to find and identify information which might have been left out, such as the back of a document that has only a stamp or small notation. All such information has to be captured. (Higher levels are reviewed if quality concerns surface during review.) The partner corrects any omissions found in the review. Skipped pages are imaged and inserted into the images folder at the correct location.
QC relating to transfer of digital materials to NARA – The partners send the digital materials to NARA on hard drives. NARA staff checks a sample of the images and metadata to verify that the metadata on each hard drive is associated with the correct image and that the metadata the partner agreed to provide is delivered. The staff also checks a sample of the unique identifiers associated with each image to verify that the identifiers are correct. If there are problems with the metadata or images sent by the partner, NARA contacts the partner to resolve the problems.
Based on both internal and external feedback, our quality control processes are under review.
Thank you to all of the researchers who have taken the time to provide feedback on the renewal of the Ancestry partnership agreement.
More than one person at NARA has now stated that some of the early 2007 “embargoed records” are currently available on the NARA website. They are NOT! If they are, someone at NARA would be able to give us the URL so we could see those early widows’ pensions ourselves. Instead, NARA continues to prohibit free access to the most profitable data sets to all except the Mormons.
Thank you for you taking the time to comment, Ms. Reeves. NARA has, in fact, made available unembargoed partner images available via the National Archives Catalog. Over 1.4 million images have been added to the series “Carded Records Showing Military Service of Soldiers who Fought in Confederate Organizations, 1903-1927” (https://catalog.archives.gov/id/586957). Click on the hyperlinked “232782 file unit(s) described in the catalog” in the “Includes” field to access the images.
Additionally, staff have uploaded 474,157 images to the series “Carded Records Showing Military Service of Soldiers who Fought in Volunteer Organizations During the American Civil War, 1890-1912” (https://catalog.archives.gov/id/300398) which should be available in the catalog in the coming weeks.
I will follow up on the early widows’ pensions to see when that it is slated to be available in the catalog.
Our staff has done considerable work in the last year to prepare metadata in order to upload partner images as well as to scale the catalog so that it can handle the increasing volume of images – both from our partners and from our internal digitization efforts – and make them accessible to all of NARA’s customers.
Denise Henderson
Digitization Division
Ms. Henderson: Thank you for posting links to digital images. Are these the only available images from the digitizing partnerships that are available from the NARA website. I found GA Confederate units and PA Union units. Perhaps I am not understanding how to access the many images that you reference. Please explain how to get to the catalog listings for the remainder.
NARA Blog Response:
I would like to respond to the quality control issue addressed in the NARA response:
What quality control measures are in place to address indexing quality, image quality, and scanning accuracy?
What quality control measures are in place to address indexing quality, image quality, and scanning accuracy?
Our quality control procedures currently are:
QC of imaging is the responsibility of the partner, following standards reported to, and approved by, NARA. The precise standards are proprietary information.
QC of metadata is the responsibility of the partner, following standards reported to, and approved by, NARA. The precise standards are proprietary information.
QC of content is the responsibility of NARA – Specifically, NARA does a page-by-page review against a five percent sample of the original records to find and identify information which might have been left out, such as the back of a document that has only a stamp or small notation. All such information has to be captured. (Higher levels are reviewed if quality concerns surface during review.) The partner corrects any omissions found in the review. Skipped pages are imaged and inserted into the images folder at the correct location.
QC relating to transfer of digital materials to NARA – The partners send the digital materials to NARA on hard drives. NARA staff checks a sample of the images and metadata to verify that the metadata on each hard drive is associated with the correct image and that the metadata the partner agreed to provide is delivered. The staff also checks a sample of the unique identifiers associated with each image to verify that the identifiers are correct. If there are problems with the metadata or images sent by the partner, NARA contacts the partner to resolve the problems.
Based on both internal and external feedback, our quality control processes are under review.
———————–
My Response:
Based on your response I understand clearly that NARA has NO Quality Control of indexing or image quality because you state that is the responsibility of the partners and is proprietary information. Why would the guardian of our nation’s most precious records give control to outside businesses and organizations with their proven record with indexing inaccuracy and poor image quality?
This same concern has been communicated to NARA staffers and archivists for several years but nothing has changed. Why not? NARA knows there are problems and has not responded. The indexes available are frequently worthless. How does making these records available in digital format help the researcher if one cannot access them. After seeing the high color scans available, I am very dissatisfied with the poor quality black and white scans available on the commercial and non-profit sites. NARA is partner in allowing the tax payers to be defrauded.
—————————
NARA Blog Response:
The embargo period is over for some partner images. Where are those images in NARA’s online catalog?
In 2013, NARA posted the first 250,000 partner images that could be released from the 2007 digitization year. Since then, we have continued to add images and additional capacity to handle the tremendous growth of the National Archives Catalog. There are currently over 5.25 million images in the Catalog. A significant proportion of these images come from our partner projects with new images added to the catalog on an ongoing basis.
—————————–
My Response:
In regard to the digital records being available after a five year embargo, please do post links so we may find them.
I have continued to search for digital records shared by the partners. Ohio, West VA, GA….where are the others?
Okay, so supposedly the Confederate compiled service records as well as the Union service records for the microfilm groups listed are searchable. Maybe I just don’t know how to search properly. Maybe you could help me out. I have personally looked at service records for quite a few Confederate soldiers named “Chronister” on the microfilm at NARA, particularly in the state of Arkansas, so I know they exist. NONE of them come up when I search through the link provided. Likewise, there are a lot of Union Chronister soldiers, and none of them come up on the other link.
Just how many rolls of this microfilm have been put online at this point? The catalog lists the entire collection of microfilmed service records for Union and Confederate, but it looks to me like NARA has barely started to put any of it in these databases. Just how much of it is here?
I am sure NARA has done a lot of work to get just this much online, but why do we continue to be told untruths about what all is there? And if the original “embargo period” was for five years, then why didn’t NARA use those five years to prepare ahead of time so it could be ready to put these images online?
I do expect NARA to allow me to see any of those early “widows’ pensions” the next time I request one in person, even though I am not a Mormon, because to refuse me at this point would further prove the religious favoritism and discrimination that has already existed for 8 years.
I missed the public comment time, I don’t know how, but I did. I’m sorry to hear that the partnership with Ancestry will continue. I had been a long time subscriber but left when errors in my own family relations were noticed, and reported, and all I got back was a one line “thanks for your interest.” Interest? these are family members whose true name is Dunleavy but is recorded as Hunleavy and yet the original census form clearly states Dunleavy. To this day, years later, the error has not been rectified. If they don’t want to believe me, just look at the census form. But no.
Ancestry has spread itself too thin, with tv shows and such and the people left wanting are the ones who pay the subscription fee and try to research their families. Ancestry could care less.