(in English also)
It seems there are two Facebook games whose the aim is precisely to
Sun, 20 Feb 2011 22:16:15 +0100, Seb35 <seb35wikipedia(a)gmail.com> wrote:
I saw VIGNERON and Jean-Frédéric today and we spoke about that.
Jean-Fred and I are a bit skeptical about the effective implementation
of such a system, here are some questions that I (or we) were asking:
(the questions are listed by order of importance.)
- how much books have such coordinates? I know the Bnf-partnership-books
have such coordinates because originally in the OCR files (1057 books),
but on WS a lot of books have non-valid coordinates (word 0 0 1 1 "")
because Wikisourcians didn't know what was the meaning of these figures
(DjVu format is quite difficult to understand anyway); I don't know if
classical OCR have a function to indicate the coordinates of future
- what is the confidence in the coordinates? if you serve an half-word,
it will be difficult to recognize the entire word
- I am asking how you can validate the correctness of a given word for a
given person: a person (e.g.) creates an account on WS, a Captcha is
asked with a word, how do you know if his/her answer is correct? I
aggree this step disapears if you ask a pool of volunteers to answer to
differents captcha-word, but in this cas it resumes to the classical
check of Wikisourcians in a specialized way to treat particular cases
- you give the example of a ^ in a word, but how do you select the
OCR-mistakes? althought this is not really an issue since you can yet
make a list of current mistakes and it will be sufficient in a first
time. I know French Wikisourcians (at least, probably others also)
already make a list of frequent mistakes ( II->Il, 1l->Il, c->e ...),
sometimes for a given book (Trévoux in French of 1771 it seems to me).
But I know Google had a similar system for their digitization, but I
don't know exactly the details. For me there are a lot of details which
makes the global idea difficult to carry out (although I would prefer
think the contrary), but perhaps has you some answers.
PS: I had another idea in a slightly different application field
(roughtly speaking automated validation of texts) but close of this one,
I write an email next week about that (already some notes in
Sat, 05 Feb 2011 15:14:57 +0100, Andrea Zanni <zanni.andrea84(a)gmail.com>
> Dear wikisourcers,
> while exploring the djvu text layer, it.source community found
> features that is good thing to share (SPOILER ALERT: Wikisource
> (I added the technicalities in the footnotes, please look at them if
> We discovered that when the text layer is extracted with djvuLibre
> djvused.exe tool 
> a text file is obtained, containing words and their absolute coordinates
> into the image of the page.
> Here a some example rows of such txt file from a running test:
> (line 402 2686 2424 2757
> (word 402 2699 576 2756 "State.")
> (word 679 2698 892 2757 "Effects")
> (word 919 2698 991 2756 "of")
> (word 1007 2697 1467 2755 "Domestication")
> (word 1493 2698 1607 2755 "and")
> (word 1637 2697 1910 2757 "Climate.")
> (word 2000 2698 2132 2756 "The")
> (word 2155 2686 2424 2754 "Persians^"))
> As you can see, the last word has a ^ character inside, that indicates a
> doubtful, unrecognized character by OCR software.
> What's really interesting is that python script can select these words
> the ^ character and produce automatically a file with the image of the
> since all needed parameters for a ddjvu.exe call can be obtained (please
> consider that this code comes from a rough, but *running* test script
> So, in our it.source test script, a tiff image has been automatically
> produced, exactly contaning the image of "Persians^" doubtful OCR
> Its name is built as name-of-djvu-file+page number+coordinates into the
> page, that it is all what is needed to link unambiguously the image and
> specific word into a specific page of a djvu file.
> The image has been uploaded into Commons as
> As you can easily imagine, this could be the core of a "wikicaptcha"
> (as John Vandenberg called it), enabling us to produce our own
> A djvu file could be uploaded into a server (into an "incubator"); a
> database of doubtful word images could be built; images could be
> to wiki users (both as a voluntary task or as a formal reCAPTCHA to
> edits by unlogged contributors); resulting human interpretation could be
> validated somehow (i.e. by n repetitions of matching, different
> interpretations) then used to edit text layer of djvu file. Finally the
> edited djvu file could be uploaded to Commons for formal source
> Please contact us if you like to have a copy of running, test scripts.
> There's too a shared Dropbox folder with the complete environment where
> are testing scripts.
> Opinions, feedbacks or thoughts are more than welcome.
> Admin it.source
> WMI Board
>  command='djvused name-of.file.djvu -e "select page-number;
> print-txt" >text-file.txt
>  if "^" in word:
> #print coord
> command="ddjvu "+fileDjvu+"
> -format=tiff "+segment+" "+filetiff
> print command