[Wikimedia-l] Re: ChatGPT as a reliable source

First: Wikipedia style for dense inline citations is one of the most granular and articulate around, so we're pushing the boundaries in some cases of research norms for clarity in sourcing. That's great; also means sometimes we are considering nuances that may be new. Second: We're approaching a topic close to my heart, which is distinguishing reference-sources from process-sources. Right now we often capture process sources (for an edit) in the edit summary, and this is not visible anywhere on the resulting article. Translations via a translate tool; updates by a script that does a particular class of work (like spelling or grammer checking); applying a detailed diff that was workshopped on some other page. An even better interface might allow for that detail to be visible to readers of the article [w/o traversing the edit history], and linked to the sections/paragraphs/sentences affected. I think any generative tools used to rewrite a section or article, or to produce a sibling version for a different reading-level, or to generate a timeline or other visualization that is then embedded in the article, should all be cited somehow. To Jimbo's point, that doesn't belong in a References section as we currently have them. But I'd like to see us develop a way to capture these process notes in a more legible way, so readers can discover them without browsing the revision history. People using generative tools to draft new material should find reliable sources for every claim in that material, much more densely than you would when summarizing a series of sources yourself. However, as we approach models that can discover sources and check facts, a combination of those with current generative tools could produce things closer to what we'd consider acceptable drafts, and at scale could generate reference works in languages that lack them. I suggest a separate project for those as the best way to explore the implications of being able to do this at scale, and should capture the full model/tuning/prompt details of how each edit was generated. Such an automatically-updated resource would not be a good reliable source, just as we avoid citing any tertiary sources, but could be a research tool for WP editors and modelers alike. SJ On Wed, May 17, 2023 at 9:27 AM Jimmy Wales <jimmywales(a)wikitribune.com> wrote: -- Samuel Klein @metasj w:user:sj +1 617 529 4266