Re: [Abstract-wikipedia] How to store wikitext along the structured content?

I'm replying to Denny's original message. I've read other replies and James's phabricator overview and I think I understand the problem. Except I don't. So I'm stepping back to the requirements and constraints. Constraint 1 is that text entered by humans must be stored as human-readable text (encoded, obviously). So, labels and aliases and "source code" are primarily text. If you need to store the text as JSON, that's fine, but I was imagining that the human text, as entered, would be translated into an abstract form (with Zs and Ks etc) and it is the abstract form that gets stored as JSON. Yes, it's dynamic (on-the-fly) during the editing process, but the human enters text we care about and the machine turns it into an object we care about. The text entered is a text and the interpreted text (object) is another text. Translations are translations of a source. You can translate the text entered into another language, and that's another text. Or you can translate the derived form, and that's a different text. But if the translation is fully automated, you might treat it as "mere" presentation: a visualisation of underlying data. Maybe it makes sense to store such a thing, especially if a human has seen it, even more so if they have relied on it, but do we have a Requirement to store all translations up-front? I don't think so. We store it in the language it was entered in (preferably with metadata that identifies the language) and maybe we store it in a small number of different languages (always having at least two would be nice). Beyond that, I think you're talking about sub-pages per language (but let's not jump to solutions). Constraint 2 is that text is bound to its context. A good example of this is comments in source-code. The word "in" indicates the binding. The comment doesn't point to some text, it is the text, right there, "in" the source code. Constraint 1 ("C1") applies: it is stored as entered, where entered. If you later want to tidy up the source code and replace comments with pointers to comments and/or translations, that's fine. But C1 still applies, so you have a new version of the source-code and you still have the old version. When it comes to documentation outside of comments, that's just a text (as written). It might be written as a multi-lingual text, but more than bi-lingual is stretching it a bit for most of us. The bi-lingual text may be a collaboration with a machine translator, but I would only see that as a Requirement when one of the languages is WMF's own synthetic language (ZKspeak, to coin a phrase). That is, I might compose my text in DeepL and paste its English into the ZObject documentation. For us, that is the text as written (C1 applies). If I also paste in the text I composed in a different language, that's fine; that's another text as written (C1 applies). (If I make a comment to that effect, that's just text where entered, but it's interesting metadata, so there may be a Requirement to capture the metadata. Either way (or both ways) C1 applies.) Requirement 1 might be that any text can be entered as Wikitext. Ah, but the JSON can't be Wikitext... Well, that isn't the Requirement. We can enter the text as Wikitext (so C1 applies). If it must be translated into text that can be JSON, that's fine. We still have the Wikitext and now we also have a translation; that's another text. Does the above guide us toward a Solution? Well, it's not A, because we don't have many translations in the JSON blob. But maybe we have three: the source human text, the interpreted ZK text and a translation into a second language. It's not B (but I don't understand B). I think we do have "secondary wikitext" but it might be implemented as "primary wikitext" with secondary translations as sub-pages (somewhat optionally), as in Meta. It would be the JSON blob that would be secondary (in a logical sence): some transformation of a primary text. If the JSON needs to be primary, you can treat it that way; then its human source pretends to be "about" the primary object. It's a bit like C, but it's not a big blob and it's probably not parallel. Maybe it's a primary Meta-like wiki that is linked by common reference (ZID) to the JSON blobosphere. Well, that sounds a lot like D, but... It's not D, because the Meta-like wiki page for a ZObject is not a sub-page of a non-wiki page. Wikipedia pages are not sub-pages of their Wikidata Item's page, but you can look at them as if they are. We can link from one Wikipedia to another directly, or we can link through Wikidata. I know callable functions are a bit different but, as I said at the beginning, I don't understand the problem. Hopefully this input will still be of benefit to somebody who does, however. Best regards, Al. On Wednesday, 29 July 2020, <abstract-wikipedia-request(a)lists.wikimedia.org> wrote: