The topics of real-time fact-checking services and AI dialogue systems are broached and
some opinions are shared for purposes of discussion.
Crowdsourced Fact-checking Resources
Are crowdsourced real-time fact-checking services possible? What might the design criteria
be for such resources? Can such resources be of use to AI dialogue systems?
Determining the truth value of a given statement – or selecting one statement from a set
of mutually exclusive statements – might not always be possible for a crowd at an instant.
The instantaneous state of and, in some cases, resolution of, fact-checking processes
might be such that multiple mutually-exclusive alternative statements are each supported
by evidence and argumentation.
Crowdsourced real-time fact-checking resources should facilitate evidence-based
argumentation for elements of sets of mutually exclusive statements as a component of
providing groups with processes to determine the factuality of statements. There may be
other means of clustering statements beyond mutual exclusivity.
Crowdsourced real-time fact-checking resources should support non-anonymous users as well
as verified users, non-anonymous users who, for example, are journalists or
Crowdsourced real-time fact-checking resources should allow users to opt into receiving
notifications whenever a particular statement or statement-cluster is updated and AI
dialogue systems should similarly be able to subscribe to such events.
Crowdsourced real-time fact-checking resources should support machine-utilizable output
formats, output formats beyond HTML.
Efficiencies and Optimizations Regarding Dialogue Systems and Knowledgebases with Remote
At an instant, does a given topic require a dialogue system to poll one or more real-time
fact-checking resources? Not every topic that a dialogue system might speak about would
seem to require polling the data of one or more fact-checking resources. If dialogue
systems could determine which topics were volatile, contentious or dynamic, and if these
determinations were cacheable for durations, then dialogue systems could more efficiently
provide their services at scale while making use of real-time fact-checking services.
It might be possible to determine the volatility, contentiousness, or dynamicism of a
given topic by checking on or receiving notifications with respect to the quantity of
recent activity for a topic on one or more fact-checking resources. Dialogue systems could
asynchronously seek or receive updates regarding this data to best produce responses for
Dialogue System Behaviors
Which behaviors should dialogue systems exhibit if asked about volatile, contentious or
dynamic topics? For instance, topics of some unfolding discussion on one or more
These scenarios might arise as users ask dialogue systems questions about topics recently
discussed or debated, e.g. in the news.
Some options are indicated:
Option 1: I Don’t Know
A dialogue system could indicate that it doesn’t know the answer.
Option 2: I Need More Time to Answer that Question
A dialogue system could indicate that the matter is being reviewed by fact-checkers and
somehow provide an estimate of when it should be able to answer the question. A dialogue
system could ask a user if they desire a notification, including within specified hours,
with respect to updates and could provide notifications to users via one of a number of
Option 3: Refer Users to Fact-checking Resources
A dialogue system could refer the user to Web materials, for example to the unfolding
discussion on a fact-checking resource. Multimodal dialogue systems could present users
with Web content.
Option 4: Explain Alternatives and Possibilities
A dialogue system could, in particular when a fact-checking resolution appears to be
disagreement, explain the elements of a set of mutually exclusive alternatives and provide
the supporting argumentation for each. “Some are indicating X, while others Y.” Questions
include how best to synthesize the output rhetoric in these instances, how best to sort
the alternatives and possibilities, how best to specify which parties were supporting
which alternatives and possibilities, and so forth.
Option 5: Decide
A dialogue system could review the argumentation and evidence supporting each element of a
set of mutually exclusive statements and make a decision.
The topics of real-time fact-checking services and AI dialogue systems were broached and
some opinions were shared for purposes of discussion.