Hi Lauren,
Thanks for attending the Future Audiences community call this week, and apologies that we
ran out of time before I was able to answer your question about the ChatGPT plugin next
steps – answering now :)
You wrote:
“As for the Foundation’s ChatGPT plugin, I’m afraid I find it mostly unusable because it
ignores everything after the first dozen paragraphs of all articles. That was listed as
needing 3-4 days to fix on
https://phabricator.wikimedia.org/T343932 a month ago. Do you
know whether there are any plans to go ahead with that fix?”
First: thank you for enabling the plugin and trying it out! We've just added a link to
a survey that we're hoping will give us more feedback on whether/how the current
plugin is meeting user needs. If you haven't already done so, I'd appreciate if
you could take a few minutes to fill it out (you should see it in the footer of the plugin
response).
On your specific point about the 12-paragraph cutoff: yes, we recognize that this may lead
to the plugin not finding relevant information in some cases (though from our qualitative
coding of about 300 query responses in 6 languages, the plugin was able to return relevant
information about 84% of the time[1]). However, as I tried to stress in my blog post,[2]
this plugin is intended to be a quick experiment and not a fully-featured, permanently
maintained product. We're tracking many possible optimizations, such as changes to the
12-paragraph cutoff, showing references, improving output quality, etc. But in order to go
deep on optimization, we first need to get compelling evidence that we *should* invest in
a product like this, on an external platform that we don't manage, longterm – because
making that investment would require getting more resources (i.e., an actual feature
development team, not just part-time R&D), which would be a nontrivial change to our
annual plan. As the WMF staffer who would be making the case for that investment
internally and to the community: I think while there are definitely more things we can use
the plugin to learn, and it's always possible that usage of ChatGPT and the plugin may
take off wildly, I don't personally feel comfortable making that recommendation at
this moment. (All that said, we may be able to get some more ad hoc R&D resources to
make some optimization tweaks as we try to learn more – that's what I'm currently
aiming for, so stay tuned!)
Please let me know if you have more questions or want to talk about plugin specifics,
on-list, via email, or onwiki (the project is on Meta here:
https://meta.wikimedia.org/wiki/Talk:Future_Audiences)!
Best,
Maryana
1.
https://meta.wikimedia.org/wiki/Future_Audiences/Experiments:_conversationa…
2.
https://diff.wikimedia.org/2023/07/13/exploring-paths-for-the-future-of-fre…