Introduction
The topics of real-time fact-checking services and AI dialogue systems are broached and some opinions are shared for purposes of discussion.
Crowdsourced Fact-checking Resources
Are crowdsourced real-time fact-checking services possible? What might the design criteria be for such resources? Can such resources be of use to AI dialogue systems?
Determining the truth value of a given statement – or selecting one statement from a set of mutually exclusive statements – might not always be possible for a crowd at an instant. The instantaneous state of and, in some cases, resolution of, fact-checking processes might be such that multiple mutually-exclusive alternative statements are each supported by evidence and argumentation.
Crowdsourced real-time fact-checking resources should facilitate evidence-based argumentation for elements of sets of mutually exclusive statements as a component of providing groups with processes to determine the factuality of statements. There may be other means of clustering statements beyond mutual exclusivity.
Crowdsourced real-time fact-checking resources should support non-anonymous users as well as verified users, non-anonymous users who, for example, are journalists or fact-checkers.
Crowdsourced real-time fact-checking resources should allow users to opt into receiving notifications whenever a particular statement or statement-cluster is updated and AI dialogue systems should similarly be able to subscribe to such events.
Crowdsourced real-time fact-checking resources should support machine-utilizable output formats, output formats beyond HTML.
Efficiencies and Optimizations Regarding Dialogue Systems and Knowledgebases with Remote Fact-checking Services
At an instant, does a given topic require a dialogue system to poll one or more real-time fact-checking resources? Not every topic that a dialogue system might speak about would seem to require polling the data of one or more fact-checking resources. If dialogue systems could determine which topics were volatile, contentious or dynamic, and if these determinations were cacheable for durations, then dialogue systems could more efficiently provide their services at scale while making use of real-time fact-checking services.
It might be possible to determine the volatility, contentiousness, or dynamicism of a given topic by checking on or receiving notifications with respect to the quantity of recent activity for a topic on one or more fact-checking resources. Dialogue systems could asynchronously seek or receive updates regarding this data to best produce responses for users.
Dialogue System Behaviors
Which behaviors should dialogue systems exhibit if asked about volatile, contentious or dynamic topics? For instance, topics of some unfolding discussion on one or more fact-checking resources?
These scenarios might arise as users ask dialogue systems questions about topics recently discussed or debated, e.g. in the news.
Some options are indicated:
Option 1: I Don’t Know
A dialogue system could indicate that it doesn’t know the answer.
Option 2: I Need More Time to Answer that Question
A dialogue system could indicate that the matter is being reviewed by fact-checkers and somehow provide an estimate of when it should be able to answer the question. A dialogue system could ask a user if they desire a notification, including within specified hours, with respect to updates and could provide notifications to users via one of a number of communication channels.
Option 3: Refer Users to Fact-checking Resources
A dialogue system could refer the user to Web materials, for example to the unfolding discussion on a fact-checking resource. Multimodal dialogue systems could present users with Web content.
Option 4: Explain Alternatives and Possibilities
A dialogue system could, in particular when a fact-checking resolution appears to be disagreement, explain the elements of a set of mutually exclusive alternatives and provide the supporting argumentation for each. “Some are indicating X, while others Y.” Questions include how best to synthesize the output rhetoric in these instances, how best to sort the alternatives and possibilities, how best to specify which parties were supporting which alternatives and possibilities, and so forth.
Option 5: Decide
A dialogue system could review the argumentation and evidence supporting each element of a set of mutually exclusive statements and make a decision.
Conclusion
The topics of real-time fact-checking services and AI dialogue systems were broached and some opinions were shared for purposes of discussion.
Best regards, Adam Sobieski
Hi,
On Fri, 2020-02-28 at 08:11 +0000, Adam Sobieski wrote:
The topics of real-time fact-checking services and AI dialogue systems are broached and some opinions are shared for purposes of discussion.
I could not find the string "wiki" in your message. Can you please elaborate what this posting has to do with "discussing the technical organization of the Wikimedia projects"?
Conclusion
The topics of real-time fact-checking services and AI dialogue systems were broached and some opinions were shared for purposes of discussion.
Passive voice can be extremely unclear by nature, I guess... Do you have some question, or why was this message posted? :)
Cheers, andre
Andre,
I elaborated on these ideas here: https://www.linkedin.com/pulse/dialogue-systems-crowdsourced-real-time-fact-... .
With respect to crowdsourced fact-checking resources, Wikipedia, Wikinews and Wikidata are relevant projects. A new sister project, Wikifact, is also possible.
It wasn’t my intention to post off-topic. I was hoping that crowdsourced real-time fact-checking resources and their interoperation with dialogue systems (e.g. Alexa, Cortana, Google Assistant, and Siri) would be of interest to some Wiki technology enthusiasts and developers.
Best regards, Adam
From: Andre Klappermailto:aklapper@wikimedia.org Sent: Friday, February 28, 2020 6:45 AM To: Wikimedia developersmailto:wikitech-l@lists.wikimedia.org Subject: Re: [Wikitech-l] Real-time Fact-checking Services and Dialogue Systems
Hi,
On Fri, 2020-02-28 at 08:11 +0000, Adam Sobieski wrote:
The topics of real-time fact-checking services and AI dialogue systems are broached and some opinions are shared for purposes of discussion.
I could not find the string "wiki" in your message. Can you please elaborate what this posting has to do with "discussing the technical organization of the Wikimedia projects"?
Conclusion
The topics of real-time fact-checking services and AI dialogue systems were broached and some opinions were shared for purposes of discussion.
Passive voice can be extremely unclear by nature, I guess... Do you have some question, or why was this message posted? :)
Cheers, andre -- Andre Klapper (he/him) | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/
_______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org