Classic Wikisource issues
-------- Messaggio inoltrato --------
Oggetto: Can You Help us Make the 19th Century Searchable?
Data: Fri, 21 Aug 2020 20:32:17 +0000
Mittente: Brewster Kahle
<http://blog.archive.org/2020/08/21/can-you-help-us-make-the-19th-century-searchable/>
Can You Help us Make the 19th Century Searchable?
In 1847, Frederick Douglass started a newspaper
<https://archive.org/details/pub_frederick-douglass-paper> advocating
the abolition of slavery that ran until 1851. After the Civil War,
there was a newspaper for freed slaves, the Freedmen’s Record
<https://archive.org/details/pub_freedmens-record>. The Internet
Archive is bringing these and many more works online for free public
access. But there’s a problem:
Our Optical Character Recognition (OCR), while the best commercially
available OCR technology, is not very good at identifying text from
older documents.
Take for example, this newspaper from 1847. The images
<https://archive.org/details/sim_frederick-douglass-paper_1847-12-03_1_1>
are not that great, but a person can read them:
The problem is our computers’ optical character recognition tech gets
it wrong
<https://archive.org/stream/sim_frederick-douglass-paper_1847-12-03_1_1/sim_frederick-douglass-paper_1847-12-03_1_1_djvu.txt>,
and the columns get confused.
What we need is “Culture Tech” (a riff on fintech, or biotech) and
Culture Techies to work on important and useful projects–the things we
need, but are probably not going to get gushers of private equity
interest to fund. There are thousands of professionals taking on similar
challenges in the field of digital humanities and we want to complement
their work with industrial-scale tech that we can apply to cultural
heritage materials.
One such project would be to work on technologies to bring 19th-century
documents fully digital.. We need to improve OCR to enable full text
search, but we also need help segmenting documents into columns and
articles. The Internet Archive has lots of test materials and thousands
are uploading more documents all the time.
What we do not have is a good way to integrate work on these projects
with the Internet Archive’s processing flow. So we need help and ideas
there as well.
Maybe we can host an “Archive Summer of CultureTech” or something…Just
ideas. Maybe working with a university department that would want to
build programs and classes around Culture Tech… If you have ideas or
skills to contribute, please post a comment here or send an email to
info(a)archive.org with some of this information.
The post Can You Help us Make the 19th Century Searchable?
<http://blog.archive.org/2020/08/21/can-you-help-us-make-the-19th-century-searchable/>
appeared first on Internet Archive Blogs <http://blog.archive.org>.