Hey Aleksey,

The library allows you to access the position of the DumpReader and to resume from a stored position.

In the docs: https://github.com/JeroenDeDauw/JsonDumpReader#resume-reading-from-a-previous-position

PHP interface: https://github.com/JeroenDeDauw/JsonDumpReader/blob/master/src/SeekableDumpReader.php

This functionality is used by Replicator, a CLI tool build on top of the JSON Dump Reader library.

Replicator: https://github.com/JeroenDeDauw/Replicator#replicator

Aborting and resuming imports with Replicator: https://github.com/JeroenDeDauw/Replicator#importing-extracted-json-dumps

Neither the library or the CLI tool support streaming dumps (unless that somehow magically ends up working). I'm happy to review pull requests with additions or enhancements.

Note that Replicator supports import from the Wikidata web API, including automatic fetching of dependencies. This works if you want to get a specific set of entities or just a few thousand for testing purposes. If you want all entities from Wikidata this approach is of course not viable.

https://github.com/JeroenDeDauw/Replicator#importing-from-the-wikidataorg-api

Cheers

--
Jeroen De Dauw | https://entropywins.wtf | https://keybase.io/jeroendedauw

Software Crafter | Speaker | Student | Strategist | Contributor to Wikimedia and Open Source
~=[,,_,,]:3

_______________________________________________
Wikidata-tech mailing list
Wikidata-tech@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-tech