There was recently a brief discussion on the enwiki technical village pump about the following issue, where Tim Starling suggested this should be fixed in the API [1]. I want to mention it here before filing a bug.
It's already known among the developers that there can be obsolete entries in the page table. One of these at the moment is pageid 414115 on enwiki, the article [[/.]], which has an illegal title. Unsurprisingly, when that page was last edited, it was a redirect to [[Slashdot]].
If I fetch the list of backlinks to [[Slashdot]] using the API, I get that page among the results. If I then try to fetch the page content for it (using action=query and prop=revisions) the API returns a bad title error ('invalidtitle'). This is of course a bad situation for people using the API.
Strangely, I can get the content of that page using a generator query [2].
- Carl
1. http://en.wikipedia.org/w/index.php?title=Wikipedia:Village_pump_%28technica...
2. http://en.wikipedia.org/w/api.php?action=query&generator=backlinks&g... The parameters are: ( 'action' => 'query', 'generator' => 'backlinks', 'titles' => 'Slashdot', 'gblfilterredir'=> 'redirects', 'prop' => 'revisions', 'rvprop' => 'ids|user|comment|content' );
Carl Beckhorn schreef:
It's already known among the developers that there can be obsolete entries in the page table. One of these at the moment is pageid 414115 on enwiki, the article [[/.]], which has an illegal title. Unsurprisingly, when that page was last edited, it was a redirect to [[Slashdot]].
That's the database's fault, not the API's. If this causes trouble, someone should write a maintenance script to delete these ghost pages.
If I fetch the list of backlinks to [[Slashdot]] using the API, I get that page among the results. If I then try to fetch the page content for it (using action=query and prop=revisions) the API returns a bad title error ('invalidtitle'). This is of course a bad situation for people using the API.
The same is true for the UI: if you go to Special:Whatlinkshere/Slashdot, you'll see that [[/.]] is listed (and linked), but clicking the link turns up an error message.
Strangely, I can get the content of that page using a generator query [2]
That's because generators are based on page IDs: list=backlinks passes a list of page IDs to prop=revisions internally.
Roan Kattouw (Catrope)
"Roan Kattouw" roan.kattouw@home.nl said:
Carl Beckhorn schreef:
It's already known among the developers that there can be obsolete entries in the page table. One of these at the moment is pageid 414115 on enwiki, the article [[/.]], which has an illegal title. Unsurprisingly, when that page was last edited, it was a redirect to [[Slashdot]].
That's the database's fault, not the API's. If this causes trouble, someone should write a maintenance script to delete these ghost pages.
Well, I'm the one who posted about this on enwiki, and was told in no uncertain terms by Tim Starling that it's the API's fault, not the database's.
The problem is that if the invalid title retrieved from one query is used in the titles= parameter of another query, the API returns a Bad Title error on the entire query (even if there are 50 titles |'ed together). Some possible solutions to this, in no particular order:
1) delete the damn page entry from the database 2) change the API so that, if there is a bad title in the titles= parameter, it still returns the query result for all the valid titles. 3) change the API so that it doesn't return the invalid page entry in the first place (probably not such a good idea, now that I think about it)
Russ
Russell Blau schreef:
"Roan Kattouw" roan.kattouw@home.nl said:
Carl Beckhorn schreef:
It's already known among the developers that there can be obsolete entries in the page table. One of these at the moment is pageid 414115 on enwiki, the article [[/.]], which has an illegal title. Unsurprisingly, when that page was last edited, it was a redirect to [[Slashdot]].
That's the database's fault, not the API's. If this causes trouble, someone should write a maintenance script to delete these ghost pages.
Well, I'm the one who posted about this on enwiki, and was told in no uncertain terms by Tim Starling that it's the API's fault, not the database's.
Even the big guys make a mistake here and there (although they do so less frequently), they're human too. The point here is that the UI has exactly the same flaw ([[Special:Whatlinkshere/Slashdot]]) also links to [[/.]], which, when clicked, displays an invalid title error). The fact that these titles are still in the DB is not the API's fault (nor the UI's, BTW).
The problem is that if the invalid title retrieved from one query is used in the titles= parameter of another query, the API returns a Bad Title error on the entire query (even if there are 50 titles |'ed together).
That's bug 13390 [1], only very distinctly related to this issue.
Some possible solutions to this, in no particular order:
- delete the damn page entry from the database
That should happen. Someone (not me, I suck at that) should write a maintenance script for that.
- change the API so that, if there is a bad title in the titles=
parameter, it still returns the query result for all the valid titles.
That's what I'm gonna do when I fix bug 13390 (it's on my TODO list).
- change the API so that it doesn't return the invalid page entry in the
first place (probably not such a good idea, now that I think about it)
Exactly.
Roan Kattouw (Catrope)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Roan Kattouw wrote:
Some possible solutions to this, in no particular order:
- delete the damn page entry from the database
That should happen. Someone (not me, I suck at that) should write a maintenance script for that.
cleanupTitles.php & co -- been there for a million years. Currently not all tables in all places are well covered by a cleanup script, though.
In general, due to the delays between changes of what's valid or fixes of buggy generating code and subsequent cleanup, you should always assume that the DB may contain "ghost" data.
- change the API so that, if there is a bad title in the titles=
parameter, it still returns the query result for all the valid titles.
That's what I'm gonna do when I fix bug 13390 (it's on my TODO list).
It's always best to handle invalid data gracefully, especially when it's easy to encapsulate like this.
Generally we assume that titles we find in the db will be valid, except when they're not. ;) That means you can treat them as valid for purposes of, say, optimizing creation of objects (with Title::makeTitle), but you should still watch out for surprises.
Sometimes you'll end up with a null when you expected an object, and it's wise to pass it over and render out the rest of the list intact. Sometimes this means putting in a placeholder results ("Invalid title!"), sometimes just dropping it from a list is better.
- -- brion vibber (brion @ wikimedia.org)
Brion Vibber schreef:
- change the API so that, if there is a bad title in the titles=
parameter, it still returns the query result for all the valid titles.
That's what I'm gonna do when I fix bug 13390 (it's on my TODO list).
It's always best to handle invalid data gracefully, especially when it's easy to encapsulate like this.
Generally we assume that titles we find in the db will be valid, except when they're not. ;) That means you can treat them as valid for purposes of, say, optimizing creation of objects (with Title::makeTitle), but you should still watch out for surprises.
Sometimes you'll end up with a null when you expected an object, and it's wise to pass it over and render out the rest of the list intact. Sometimes this means putting in a placeholder results ("Invalid title!"), sometimes just dropping it from a list is better.
I intend to change the API to handle invalid titles analogous to missing titles, i.e. <page ns="0" title="/." invalid="" />
Roan Kattouw (Catrope)
Roan Kattouw schreef:
I intend to change the API to handle invalid titles analogous to missing titles, i.e. <page ns="0" title="/." invalid="" />
Done in r32109 [1], see bug 13390 comment #1 [2] for example output.
Roan Kattouw (Catrope)
[1] http://svn.wikimedia.org/viewvc/mediawiki?view=rev&revision=32109 [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=13390#c1
On Mon, Mar 17, 2008 at 03:56:48PM -0400, Russell Blau wrote:
"Roan Kattouw" roan.kattouw@home.nl said:
That's the database's fault, not the API's. If this causes trouble, someone should write a maintenance script to delete these ghost pages.
Well, I'm the one who posted about this on enwiki, and was told in no uncertain terms by Tim Starling that it's the API's fault, not the database's.
The existence of the invalid titles is the database's fault. However, I seem to recall from past experience that the general philosophy is for Mediawiki to handle them gracefully. That's the light in which I read Tim's comments.
- change the API so that, if there is a bad title in the titles=
parameter, it still returns the query result for all the valid titles.
There is an open bug report about that:
https://bugzilla.wikimedia.org/show_bug.cgi?id=13390
- Carl
mediawiki-api@lists.wikimedia.org