Dear list,
we have included parts of wikibooks (languages with most content) in our academic search engine. The pages are generated by getting title, url and abstract from wikibooks abstract.xml This is combined with pages-arcticles.xml to build metadata pages for indexing. Unfortunately, the building of wikibooks-latest-abstract.xml.gz has been discontinued last year. Currently we have 389.429 records indexed with 24 languages but they are now over a year old.
Is there any replacement for "Page abstract for Yahoo" dumps?
If not, is there any easy solution to fetch or generate these data somehow?
Kind regards Bernd
(For context to others reading this, the page abstract dumps were removed on T382069 Undeploy and archive ActiveAbstract https://phabricator.wikimedia.org/T382069 due to cost and low usage).
Bernd:
I think you could re-implement your use case by leveraging Wikimedia Enterprise's Snapshot API:
curl -L ' https://api.enterprise.wikimedia.com/v2/snapshots/enwikibooks_namespace_0/do...' \ -H 'Authorization: Bearer ACCESS_TOKEN' \ --output enwikibooks.tar.gz
That will give you compressed NDJSON, and you can further parse it like so:
tar -xzf enwikibooks.tar.gz # Each line is one article; extract name, url, abstract: cat enwikibooks.ndjson | jq '{name: .name, url: .url, abstract: .abstract}'
More info at https://enterprise.wikimedia.com/project-data/wikibooks-api/
Note Enterprise is free for non-profit use cases. You do need to have an account with them to get the authorization token.
Hope this helps, -xabriel
On Mon, May 11, 2026 at 8:44 AM Bernd Fehling via Xmldatadumps-l < xmldatadumps-l@lists.wikimedia.org> wrote:
Dear list,
we have included parts of wikibooks (languages with most content) in our academic search engine. The pages are generated by getting title, url and abstract from wikibooks abstract.xml This is combined with pages-arcticles.xml to build metadata pages for indexing. Unfortunately, the building of wikibooks-latest-abstract.xml.gz has been discontinued last year. Currently we have 389.429 records indexed with 24 languages but they are now over a year old.
Is there any replacement for "Page abstract for Yahoo" dumps?
If not, is there any easy solution to fetch or generate these data somehow?
Kind regards Bernd
--
Bernd Fehling Bielefeld University Library Dipl.-Inform. (FH) LibTec - Library Technology Universitätsstr. 25 and Knowledge Management 33615 Bielefeld Tel. +49 521 106-4060 bernd.fehling(at)uni-bielefeld.de https://www.ub.uni-bielefeld.de/~befehl/
BASE - Bielefeld Academic Search Engine - www.base-search.net
Xmldatadumps-l mailing list -- xmldatadumps-l@lists.wikimedia.org To unsubscribe send an email to xmldatadumps-l-leave@lists.wikimedia.org
xmldatadumps-l@lists.wikimedia.org