Thanks, Charles. That's a very fine piece of work! And so relevant, not just to the quiz use case but to the whole of the NLG side of our project. I wonder whether there have been any further developments. I don't see any links to that pdf on Google. I've put the link in our to-do list[1].
There's some good stuff in there about performance as well as data quality, which are both areas we should certainly be looking into.
On the question of data structure, I guess it rather depends where you are sitting. The essence of my quiZiverse idea is that the consumer handles a relatively small dataset (client side) because WMF servers are running the functions. Essentially, the result is a pretty manageable structured object (in the more collaborative mode, perhaps a ZObject) and it can be grown iteratively. Given those assumptions, it hardly matters which formats are desired as inputs to (for consumption within) an actual quiz.
Since I was envisaging passing out "links" that are essentially a pre-written call back to the function with different arguments, those potential call-backs could be queued for processing server-side, so that the eventual call-back is referencing a freshly minted structured object (with a fairly limited shelf-life, unless it's a refresh of a pre-existing ZObject). So, again, lots of options to explore on the technical side.
I'm inclined to disagree with you on the question of hints, though. The structure I was envisaging is very straightforward; the "distractor" is an answer to some other question, so the "hint" is just that question phrased as a statement. It could be more complicated, but it should still be a fairly simple connection to the next function call. Of course, "where next?" may depend on whether the question was answered correctly, or there might be a choice to be made, but I think that would still resolve quite simply into a single next call. The functionality is an interactive "crawler" [2], at the end of the day, with each "next step" deferred until required or pre-prepared if responsiveness might be an issue.
Keep it simple; iterate collaboratively; make it great!
Best regards, Al. [1] https://meta.wikimedia.org/wiki/Talk:Abstract_Wikipedia/ Related_and_previous_work/Natural_language_generation https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Related_and_previous_work/Natural_language_generation
[2] https://en.wikipedia.org/wiki/Web_crawler
On Monday, 3 August 2020, abstract-wikipedia-request@lists.wikimedia.org wrote:
Send Abstract-Wikipedia mailing list submissions to abstract-wikipedia@lists.wikimedia.org
To subscribe or unsubscribe via the World Wide Web, visit https://lists.wikimedia.org/mailman/listinfo/abstract-wikipedia or, via email, send a message with subject or body 'help' to abstract-wikipedia-request@lists.wikimedia.org
You can reach the person managing the list at abstract-wikipedia-owner@lists.wikimedia.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Abstract-Wikipedia digest..."
Today's Topics:
- Re: Comprehension questions (Charles Matthews)
- Natural Language and Mathematics Generation (Adam Sobieski)
- Re: Natural Language and Mathematics Generation (Charles Matthews)
- Loose notes (Andy)
Message: 1 Date: Sun, 2 Aug 2020 17:22:11 +0100 (BST) From: Charles Matthews charles.r.matthews@ntlworld.com To: "General public mailing list for the discussion of Abstract Wikipedia (aka Wikilambda)" <abstract-wikipedia@lists.wiki media.org> Subject: Re: [Abstract-wikipedia] Comprehension questions Message-ID: 29359200.234452.1596385332016@mail2.virginmedia.com Content-Type: text/plain; charset="utf-8"
On 02 August 2020 at 16:16 Grounder UK grounderuk@gmail.com wrote:
But we don't just want the answer, we want a quiz! Equally, maybe wedon't just want the question and the answer, we want some wrong answers and some tips.
It goes back to 2016, just to generate questions from Wikidata:
https://pub.tik.ee.ethz.ch/students/2016-FS/BA-2016-03.pdf
Technically the incorrect answers in multiple choice are called "distractors". Clearly this is a rather simple data structure to handle. Hints assume quite a bit more.
At the beginning of 2017, I decided to take seriously the suggestion (from Magnus Manske) that questions should be treated as structured data. I even suggested Wikidata should have a namespace for them (this didn't go down well). A road not taken then, and just as the Comprende! tool was finished I got diverted into a Wikimedian in Residence position. So much for that.
Anyone, one take on this is that AW output might be some kind of structured data, rather than the sectioned prose (+media files and tables and templated data) familiar from Wikipedia.
By the way, mathematics in wikitext has traditionally been a threefold mix of approaches (HTML, png, LaTex): not an elegant solution.
Charles
On 03 August 2020 at 13:56 Grounder UK grounderuk@gmail.com wrote:
Keep it simple; iterate collaboratively; make it great!
I'll just remark that my thinking here dates from 2014, when a friend from the Moodle world remarked that he really would like to see a site on which questions were edited collaboratively. That presupposes some format decisions, at least if we assume that educational material evolves through a series of mashups, which I believe is historically true.
So whatever the refinements (hints, scoring, timed tests and their security, conditional branching, drag-and-drop graphics ...) the basic planks seem to be:
* An agreed set of formats that can be reused.
* Licensing that allows reuse (there is a point here about CC-by-SA, that it gives credit to those who develop good content).
* Findability by topical indexation. In fact the real-world need is to be able to find in minutes material within quite narrow criteria, "next week's test". Complex searches, such as SPARQL allows, are implied.
What AW would allow, in particular, would clearly be the fresh dimension of language independence. Hooray: big win.
We should also note this: some useful multiple choice questions are considered valuable, and exam papers containing them are not made public. This is a loss for those being educated, and so here is a key OER angle.
Charles
abstract-wikipedia@lists.wikimedia.org