Hi Eric, 

      Whenever I create a new Wikibase, the first property (P1)  I create is "Wikidata mapping" with a value data type URL. And for each Item that reflects a concept that exists in Wikidata, I add the URL for that wikidata item to the related Wikibase item. Doing so the items are now connected.

With quite some caveats it is possible to replicate the Q numbers from WIkidata in Wikibase, however, using a mapping property is IMHO a preferred and more stable solution. One of those caveats is that it is not 100% full proof. For example, if you make a copy of Wikidata (or a subset) on day one, both your Wikibase and Wikidata will have separate lives. If that is not the case why bother setting up a Wikibase not simply rely on wikidata. So if at some time later want to again sync with wikidata, you have suddenly Qids on your Wikibase (ie. items you created) that have the exact same Qid of a totally different Wikidata item. Meaning that in the long run, you will not be able to sustain the Wikidata Qid. 

When I want to query my personal wikibase for a Wikidata item that I replicated in my Wikibase, I use the following query:

"SELECT * WHERE {?item wbt:P1 wd:Q42 ; wbt:Pxx ?some_extra_annotation_not_in_wikidata } ."


On Fri, Jan 29, 2021 at 4:10 AM Erik Paulson <epaulson@unit1127.com> wrote:
Hello - 

Sorry for the crosspost/repeat - I sent a version of this to the wikidata mailing list, but it was right in the peak of the holidays. This list is probably more appropriate for it and hopefully by now the wikibase developers are back from their holidays and all caught up on email/year end/new year tasks, and can help provide some guidance. 

The tl;dr version of this post: On a blank Wikibase instance, I want to be able to do:

api.php?action=wbeditentity&new=item&id=Q42&data={"labels":{"en":{"language":"en","value":"Douglas Adams"}}}

I do not want to do this on wikidata.org - I understand why it makes no sense in that context. But I would like to be able to do this on my own Wikibase instance.

Beyond the whimsical like ensuring Doug Adams gets to be Q42, the main reason for this is data portability and identifier stability. As more hosted Wikibase providers come online and start offering services, I want to know that I have data portability if I need to change to a different provider. Anyone who queries my Wikibase needs to know the identifiers my Wikibase uses for instances and more importantly for classes, and if I change providers, those identifiers cannot change without breaking those queries. 

I do not think that MySQL backups are a reliable way to be able to transition between providers. I am not confident that all providers will want to offer a service where they accept a MySQL backup to load into their Wikibase backend, and there are additional challenges moving between Wikibase versions. (Though some may - I programmatically create the contents of my Wikibase so I don't care about edit history, but if one were to care about that history and other things like wikiusers I imagine the MySQL dumps would be the preferred way to migrate?)

One possible solution is to simply create blank items in a new Wikibase, from 1 to the maximum identifier used in my old wikibase, and then repopulate each item with the claims from my old Wikibase instance. Unfortunately this is not a reliable solution because while Wikibase guarantees that item IDs will not be reused, it does not guarantee that every ID in the sequence will be created, e.g. in rare cases Wikibase may go from Q41 to Q43 and skip/never create Q42. 

I don't mind that the identifier needs to be prefixed with a 'Q' or a 'P' for a particular type, I just want to be able to set the same identifier if I set up a new wikibase instance. 

I think Wikibase is awesome, but it is an odd database that does not allow you to set the keys for the data you are managing :)

In reading through the Wikibase Repo code, it seems like this scenario was considered though perhaps isn't fully implemented (or has been disabled?). The code in EntitySavingHelper.php looks like there are/were ways to call it by providing an ID while still asking for a new entity, though there is logic earlier in the ModifyEntity code to look for and explicitly reject the case where the API asks for 'new' and also provides an ID, so I'm not sure how this code path would get called. There is also code to ask the entityStores if they 'canCreateWithCustomId', but those all appear to just return 'false'?

However, if that logic was skipped in the API handler and a bit of code reworked in ModifyEntity and EntitySavingHelper, along with ensuring that that the next available ID is kept up to date in the wb_id_counters table to always be 1 beyond the maximum ID in use, it looks like it might not be that hard to enable creating entities with specific IDs?

So three questions:
Would the Wikibase development team ever be open to supporting something like this, behind a flag like $wgWBRepoSettings['allowUserProvidedIds'] that defaulted to false?

Are there more complicated implications from allowing a change like this that would need to be considered? I understand why the Wikidata.org repo needs this codepath fast and can't allow users to provide IDs for new entities anyway, but are there other reasons this isn't supported beyond "Wikidata doesn't need it?"

Is this all moot with the eventual REST API? I see that there's a PUT envisioned, could I use that to directly create an item or property and give it an ID then, or does the ID have to already exist to replace it?

I am happy to try to tackle creating a patch for this, but I'd like to get some feedback if there's any big lurking issues that I should know about before starting on the work - I'd rather not get deep into it only to find out it will never work or never be accepted. I'm also happy to shift this to phabricator if that's more appropriate. 

Thank you all for your work on Wikibase!


Wikibaseug mailing list