Remember that clients should not be depending on the specific query string data returned inside the query-continue node, with either the old-style or new style continuation.
As a partial fix for bug 24782[1], Gerrit change 22742 will cause action=query&list=recentchanges to start returning rccontinue for continuations rather than rcstart as it has done in the past. Changes of this type may be coming to other modules as well as further fixes to bug 24782 are implemented.
[1]: https://bugzilla.wikimedia.org/show_bug.cgi?id=24782 [2]: https://gerrit.wikimedia.org/r/#/c/22742/
Hi Brad,
Our code hardcoded "rcstart" in both string value and C++ class member name. This change is really a breaking one to us.
Could you tell me when this is released to *wiki?
(right now, I'm thinking a quick dirty fix, hope it can be done in 1-2 days, and I do want to switch to new "continue=" style, but that takes longer, 1-2 weeks, I'm afraid)
Thanks
On Thu, Mar 21, 2013 at 5:03 PM, Brad Jorsch bjorsch@wikimedia.org wrote:
Remember that clients should not be depending on the specific query string data returned inside the query-continue node, with either the old-style or new style continuation.
As a partial fix for bug 24782[1], Gerrit change 22742 will cause action=query&list=recentchanges to start returning rccontinue for continuations rather than rcstart as it has done in the past. Changes of this type may be coming to other modules as well as further fixes to bug 24782 are implemented.
-- Brad Jorsch Software Engineer Wikimedia Foundation _______________________________________________ Mediawiki-api-announce mailing list Mediawiki-api-announce@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce
On Thu, Mar 21, 2013 at 10:15 PM, Jiang BIAN bianjiang@google.com wrote:
Our code hardcoded "rcstart" in both string value and C++ class member name. This change is really a breaking one to us.
Could you tell me when this is released to *wiki?
The change is coming with 1.21wmf12. Due to a major bug in that release, I don't know when it will be deployed anywhere besides test2wiki.
Note that the "rcstart" parameter is not going away. It's just that the query-continue will start using "rccontinue" as a key rather than "rcstart". For the moment, you can compare [1] and [2] to see the difference.
[1]: https://en.wikipedia.org/w/api.php?action=query&list=recentchanges&r... [2]: https://test2.wikipedia.org/w/api.php?action=query&list=recentchanges&am...
On Fri, Mar 22, 2013 at 6:52 AM, Brad Jorsch bjorsch@wikimedia.org wrote:
On Thu, Mar 21, 2013 at 10:15 PM, Jiang BIAN bianjiang@google.com wrote:
Our code hardcoded "rcstart" in both string value and C++ class member name. This change is really a breaking one to us.
Could you tell me when this is released to *wiki?
The change is coming with 1.21wmf12. Due to a major bug in that release, I don't know when it will be deployed anywhere besides test2wiki.
Can I understand that once the bug is fixed on test2wiki, it will be deployed to *wiki and other media wiki sites immediately? or is it released on a regular base (e.g. 2 weeks later? - in this case, it means we have at least 2 weeks to fix our code)
And when deploying the new release, is it done to all sites at once or one-by-one in a release window? (if the release proces last for a while, e.g. 1-2 days, our change need to handle both new style and old style, right?)
Note that the "rcstart" parameter is not going away. It's just that the query-continue will start using "rccontinue" as a key rather than "rcstart". For the moment, you can compare [1] and [2] to see the difference.
Here is more background of our broken: (sadly) our code DO look into both the value of "rcstart" (e.g. "2013-03-23T02:53:28Z") and the attribute name itself ("rcstart"). Now I understand we should not do that.
But it is not easy for us to have a quick fix this time, the parsing of XML is done by another library which is much older, and hard to change.
To have a complete fix (I mean respect the spec, not looking into the attribute names or values), we need 1-2 weeks at least. So I'd prefer a quick fix if something still works.
It looks to me I can still sends out rcstart by strip the "|?????" from rccontinue value: e.g. https://test2.wikipedia.org/w/api.php?action=query&list=recentchanges&am...
Could you confirm this is what I can expect with the release of 1.21wmf12?
Thanks
-- Brad Jorsch Software Engineer Wikimedia Foundation
Mediawiki-api mailing list Mediawiki-api@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-api
On Thu, Mar 21, 2013 at 8:03 PM, Brad Jorsch bjorsch@wikimedia.org wrote:
Remember that clients should not be depending on the specific query string data returned inside the query-continue node, with either the old-style or new style continuation.
That this is considered backwards-compatible confuses me. Are there other keys that we should we ignore in a typical API response?
Jim
On Fri, Mar 22, 2013 at 10:21 AM, Jim Safley jimsafley@gmail.com wrote:
That this is considered backwards-compatible confuses me. Are there other keys that we should we ignore in a typical API response?
It has always been the case that clients should not be relying on the specific keys or values to be included in the query string for continuation, beyond using the prefix to determine if the parameter belongs to the generator or not in the case of a query like action=query&generator=categories&prop=categories.
In other words, it shouldn't matter whether the result contains
<query-continue> <recentchanges rcstart="2013-03-22T13:49:07Z" /> </query-continue>
or
<query-continue> <recentchanges rccontinue="2013-03-22T10:01:55Z|13804" /> </query-continue>
You should just be using whatever is on that "recentchanges" node without regard for the specific keys or values (beyond correct handling for generators).
Or, for the new style continuation, it shouldn't matter whether it's
<continue continue="-||" rcstart="2013-03-22T14:33:34Z" />
or
<continue continue="-||" rccontinue="2013-03-22T14:27:48Z|13806" />
Again, you should just be using everything from the "continue" node without regard for the specific keys or values.
On Thu, Mar 21, 2013 at 8:03 PM, Brad Jorsch bjorsch@wikimedia.org wrote:
Remember that clients should not be depending on the specific query string data returned inside the query-continue node, with either the old-style or new style continuation.
I am not certain what this means. The names of the continuation parameters have long been documented in the API spec at
http://www.mediawiki.org/wiki/API:Main_page .
For example,
http://www.mediawiki.org/wiki/API:Embeddedin
states that the continuation parameter for that module is named 'eicontinue'.
The established system was that the content of the continuation parameters is opaque, but the names of the continuation parameters are always the same. So for "old-style" continuation we were free to hard-code the string "eicontinue" into an API client because that was part of the documented spec.
If part of the change with the new continuation system is that the entire contents of the 'query-continue' node should be treated as opaque, rather than just the values of the continuation parameters, the documentation will need to be overhauled to make that more obvious.
- Carl
On Sat, Mar 23, 2013 at 9:15 AM, Carl (CBM) cbm.wikipedia@gmail.com wrote:
On Thu, Mar 21, 2013 at 8:03 PM, Brad Jorsch bjorsch@wikimedia.org wrote:
Remember that clients should not be depending on the specific query string data returned inside the query-continue node, with either the old-style or new style continuation.
I am not certain what this means. The names of the continuation parameters have long been documented in the API spec at
http://www.mediawiki.org/wiki/API:Main_page .
For example,
http://www.mediawiki.org/wiki/API:Embeddedin
states that the continuation parameter for that module is named 'eicontinue'.
Perhaps it's not clear in the documentation, but I've always written my continue functions to do something like the following, which seems to accord with what Brad is saying.
for (JSONObject moduleObject : prevResult.getJSONObject("query-continue")) for (String moduleParam : moduleObject.keys()) nextParams.put(moduleParam, moduleObject.getString(moduleParam));
This has the advantage of being query-independent, fairly simple, and not being broken when continue parameters change, documented or not.
Cheers, -Madman/ea
On Sat, Mar 23, 2013 at 9:15 AM, Carl (CBM) cbm.wikipedia@gmail.com wrote:
On Thu, Mar 21, 2013 at 8:03 PM, Brad Jorsch bjorsch@wikimedia.org wrote:
Remember that clients should not be depending on the specific query string data returned inside the query-continue node, with either the old-style or new style continuation.
I am not certain what this means. The names of the continuation parameters have long been documented in the API spec at
Not exactly. It is documented that 'uccontinue' may be used for continuation, because all possible parameters to a module are documented. But it is not documented anywhere that 'ucstart' might be returned inside the query-continue node instead, even though this could currently be the case (and note this is another candidate for bug 24782 fixing).
The established system was that the content of the continuation parameters is opaque, but the names of the continuation parameters are always the same. So for "old-style" continuation we were free to hard-code the string "eicontinue" into an API client because that was part of the documented spec.
{{citation needed}}
I see the documentation on query-continue has been moved to https://www.mediawiki.org/wiki/API:Legacy_Query_Continue as part of the zealous pushing of the "new-style" continuation, but it does state that both keys and values should not be counted on. Earlier versions don't seem to have made it clear how continuation works at all.
On Sat, Mar 23, 2013 at 11:15 PM, Brad Jorsch bjorsch@wikimedia.org wrote:
It is documented that 'uccontinue' may be used for continuation, because all possible parameters to a module are documented. But it is not documented anywhere that 'ucstart' might be returned inside the query-continue node instead, even though this could currently be the case (and note this is another candidate for bug 24782 fixing).
Exactly - putting ucstart into query-continue was already a breaking change whenever it was done, because the documentation states that uccontinue is the parameter that is used for continues.
To fix the bug 24782, what is happening is essentially that the API is going back to what it should have been doing all along, using e.g. "uccontinue" for the continue parameter instead of "ucstart".
In general, if it is possible to use the existing, documented parameter name, that will cause fewer problems than renaming it. In particular, unless there is a maximum field length to worry about, it seems like it should always be possible to honor the original continuation parameter name for each module. That gives backwards compatibility as long as the client treats the value as opaque data.
The documentation that the client should pass the entire query-continue section is very recent [1] and was not at all part of the original document API spec when it was introduced circa 2007.
- Carl
1: http://www.mediawiki.org/w/index.php?title=API:Query&diff=603120&old...
On Sat, Mar 23, 2013 at 11:40 PM, Carl (CBM) cbm.wikipedia@gmail.com wrote:
Exactly - putting ucstart into query-continue was already a breaking change whenever it was done, because the documentation states that uccontinue is the parameter that is used for continues.
That statement does not appear to be correct. ucstart was used in the very first revision of the module.[1] uccontinue wasn't added until almost two years later.[2]
Given that, the rest of your argument is based on a faulty premise. Not that it had relevance to the recentchanges module change that began this in the first place, considering that that has never had an "rccontinue" parameter until now.
The documentation that the client should pass the entire query-continue section is very recent [1] and was not at all part of the original document API spec when it was introduced circa 2007.
Have you a link to the original spec about the query-continue section? Otherwise you're left to arguing over whose interpretation of an underspecified feature is correct, which is ultimately an exercise in futility.
[1]: https://www.mediawiki.org/wiki/Special:Code/MediaWiki/17344 [2]: https://www.mediawiki.org/wiki/Special:Code/MediaWiki/42198
On Sun, Mar 24, 2013 at 11:14 AM, Brad Jorsch bjorsch@wikimedia.org wrote:
That statement does not appear to be correct. ucstart was used in the very first revision of the module.[1] uccontinue wasn't added until almost two years later.[2]
Thank you for noticing that. I was thinking of cmstart, which was added as a continuation parameter [1] when cmcontinue had already been established. As you can see, the overall API code is somewhat messy in terms of the parameter names: cmcontinue but rcstart, etc.
Have you a link to the original spec about the query-continue section?
The continuation system dates back to query.php, which would return a "next" parameter when a query needed to be continued; the value of that parameter was then used to set a specific continuation "from" parameter in order to get the next set of data. In api.php, the values of the "next" parameters were moved to the query-continue section of the response, and the "from" parameters were renamed.
I think we agree that the original system was not so well documented. Of course it is a volunteer project - Yuri asked for help with the documentation in 2007, and I was among the people who didn't spend much time helping with that. So the documentation should not be taken as a reflection on Yuri's work at the time to implement things.
But if there was never any documentation until 2012 that the entire query-continue section should be passed along (I have been unable to find it), there's no reason to think the existing clients would already be doing it. The only consequence of this is that changes to the names of the continuation parameters for the "old" system should be minimized, and the "new" system should be more clearly documented.
- Carl
1: https://www.mediawiki.org/wiki/Special:Code/MediaWiki/33256
mediawiki-api@lists.wikimedia.org